Review AI after Entity or prompt changes
Prepare common questions, false assumptions, unknown facts, and staff follow-up requests.
After this taskEach question has a result and an identified source for any correction.
Starting conditions and inputs
Verify the scope of the changed Entities and prompt.
Values for this task
entry_url, account_id, entity_ids, template_id, test_questions, expected_answers, test_conversations_url
Command sequence
open "$entry_url" && snapshot -i
get text body
# Outside browser: The test customer sends one known-answer question, one unknown-answer question and one handover request.
open "$test_conversations_url" && snapshot -i
get text bodyRef targets and expected results
| # | Command / ref | Target and verification |
|---|---|---|
| 1 | get text | Read current prompt template and Entity values. |
| 2 | Outside browser | The test customer sends one known-answer question, one unknown-answer question and one handover request. |
| 3 | open | Conversation records for the tests |
| 4 | get text | Compare answers with source Entities and expected handover conditions. |
Check the result
Each question has a result and an identified source for any correction.
Resuming after interruption
Check the original request ID and outcome first. Do not resubmit while its result remains unknown.
Prepare test cases
Prepare common questions, false assumptions, unknown facts, and staff follow-up requests. Write expected outcomes before testing. Keep reusable cases for each language and account.
Check factual answers
Compare prices, hours, and service scope with source Entities. Distinguish incorrect facts from incorrect reference scope. Check that numbers and conditions remain consistent in translated answers.
Check behavior
Test the defined answer scope, question order, ending, and handoff rules. Include unauthorized changes and missing inputs. One correct answer does not verify every path.
Decide on operating use
Record pass or fail, account, and change per test case. Correct factual or behavioral causes and repeat the same cases. During operation, review staff corrections and recurring answer errors.