chatify icon indicating copy to clipboard operation
chatify copied to clipboard
verified publisher icon ContextLab

Tests [TODO]

Open jeremymanning opened this issue 2 years ago • 12 comments

Overview

Write unit tests for each function

Completion criteria

  • [ ] For each function, add a "todo" issue to create a test for the function, describing (a) what the function is supposed to do and (b) how it should be tested. If there are any edge cases or tricky scenarios to watch out for, those should be noted too

Resources (optional)

  • pytest: framework for writing and running tests
  • GitHub actions: can be used to automate tests when pull requests are merged
  • davos: archive contains useful code for running tests that involve notebooks

Discussion points

Adding issues doesn't require coding, but it'd be good to have at least one "coding" person work on this task to help define useful tests.

jeremymanning avatar Apr 27 '23 14:04 jeremymanning

Langchain testing https://python.langchain.com/en/latest/modules/models/llms/examples/fake_llm.html?highlight=fake

HemuManju avatar May 04 '23 15:05 HemuManju

Thank you so much !

xuyi0619 avatar May 04 '23 15:05 xuyi0619

@xuyi0619, hope you are doing well! I think the project is reaching a critical mass where we need extensive testing of the modules. Can we collaborate on this and write test cases?

HemuManju avatar May 18 '23 15:05 HemuManju

Hi @HemuManju , of course！Let’s meet on zoom. I have time tomorrow, and next Tuesday and Thursday. Let me know which date and time works for you.

xuyi0619 avatar May 18 '23 15:05 xuyi0619

I am available on Tuesday.

HemuManju avatar May 18 '23 15:05 HemuManju

Ok, I will send you the zoom link when I get back.

xuyi0619 avatar May 18 '23 17:05 xuyi0619

Grace Xu is inviting you to a scheduled Zoom meeting.

Topic: Grace Xu's Personal Meeting Room

Join Zoom Meeting https://us04web.zoom.us/j/3252265589?pwd=eG1DVjRHSWtFQXFnOE5hK2NqRlgrZz09

Meeting ID: 325 226 5589 Passcode: TuXSp9

I am available on Tuesday.

xuyi0619 avatar May 20 '23 18:05 xuyi0619

Thanks @xuyi0619, at time are we meeting? I am on west coast.

HemuManju avatar May 22 '23 00:05 HemuManju

How about 1pm?

On Sun, May 21, 2023 at 8:06 PM Hemu @.***> wrote:

Thanks @xuyi0619 https://github.com/xuyi0619, at time are we meeting? I am on west coast.

— Reply to this email directly, view it on GitHub https://github.com/ContextLab/chatify/issues/9#issuecomment-1556336910, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3QX6NTEFW3PLFLN2276UJTXHKUXHANCNFSM6AAAAAAXN6ONYI . You are receiving this because you were mentioned.Message ID: @.***>

xuyi0619 avatar May 22 '23 04:05 xuyi0619

Works for me! Looking forward for the meeting.

HemuManju avatar May 22 '23 04:05 HemuManju

@HemuManju I am on zoom.

xuyi0619 avatar May 23 '23 20:05 xuyi0619

Some tests are hard to write, and others seem easier.

Hard tests

  • Making sure the interface renders correctly
  • Making sure the prompts are being sent to the LLM correctly
  • Testing the efficacy of the prompts
  • Checking formatting stuff (e.g., syntax highlighting, markdown rendering)

Easier tests

  • Verify that each type of LLM can be created without crashing, and can produce string outputs when called
  • Verify that a Chatify object can be created without crashing

jeremymanning avatar Jul 31 '23 13:07 jeremymanning