Conversation
🛡️ Jit Security Scan Results✅ No security findings were detected in this PR
Security scan by Jit
|
There was a problem hiding this comment.
Review completed. The PR adds comprehensive system testing infrastructure for production readiness validation. No high-confidence suggestions at this time.
🤖 Automated review complete. Please react with 👍 or 👎 on the individual review comments to provide feedback on their usefulness.
| ) | ||
|
|
||
| start = time.perf_counter() | ||
| response = await scale_test_client.put_working_memory( |
There was a problem hiding this comment.
The test calls put_working_memory without first creating the session. According to the test failures in test_results_full.txt, this results in 404 errors. The API appears to require sessions to be created before they can be updated.
Consider adding a session creation step before calling put_working_memory, or verify that put_working_memory should create sessions if they don't exist (and fix the implementation if needed).
# Before put_working_memory, consider:
await scale_test_client.create_session(session_id)
# Or verify put_working_memory should handle session creation| await client.put_working_memory(session_id, working_memory) | ||
|
|
||
| # Wait a bit for background summarization to potentially occur | ||
| await asyncio.sleep(2) |
There was a problem hiding this comment.
Using a fixed 2-second sleep to wait for background summarization is fragile. If the system is under load or the summarization takes longer, this test may produce false negatives. If it completes faster, the test wastes time.
Consider implementing a polling mechanism with a timeout:
# Instead of fixed sleep:
for _ in range(10): # Poll up to 10 times
await asyncio.sleep(0.5)
result = await client.get_working_memory(session_id)
if result.context: # Summarization occurred
break| ) | ||
|
|
||
| start = time.perf_counter() | ||
| response = await travel_agent_client.put_working_memory( |
There was a problem hiding this comment.
Same issue as in the scale tests - calling put_working_memory without first creating the session results in 404 errors (see test failures). This pattern is repeated throughout both test files.
Ensure sessions are created before attempting to update them, or verify the API implementation handles session creation in put_working_memory.
|
|
||
| # Check if server is running | ||
| SERVER_URL="${MEMORY_SERVER_BASE_URL:-http://localhost:8001}" | ||
| if ! curl -s -f "$SERVER_URL/health" > /dev/null 2>&1; then |
There was a problem hiding this comment.
The health check endpoint test uses curl -s -f which will fail silently if the endpoint returns a non-2xx status. However, the error message assumes the server is not reachable, which may be misleading if the server is running but returns an error status.
Consider checking the actual HTTP status or providing more specific error information:
HTTP_STATUS=$(curl -s -o /dev/null -w "%{http_code}" "$SERVER_URL/health")
if [ "$HTTP_STATUS" != "200" ]; then
echo -e "${RED}❌ Memory server health check failed (HTTP $HTTP_STATUS)${NC}"
exit 1
fi| trip = next((t for t in trips if t["trip_number"] == trip_number), None) | ||
|
|
||
| if not trip: | ||
| raise ValueError(f"Trip {trip_number} not found") |
There was a problem hiding this comment.
The ValueError is raised when a trip is not found, but the error handling doesn't account for invalid trip numbers (e.g., 0, negative numbers, or numbers > 3). While the current implementation works, consider adding validation:
if trip_number < 1 or trip_number > 3:
raise ValueError(f"Trip number must be 1-3, got {trip_number}")
trip = next((t for t in trips if t["trip_number"] == trip_number), None)
if not trip:
raise ValueError(f"Trip {trip_number} not found in test data")
No description provided.