Skip to content

feedback#2940

Open
shagun-singh-inkeep wants to merge 24 commits intomainfrom
implementation/feedback
Open

feedback#2940
shagun-singh-inkeep wants to merge 24 commits intomainfrom
implementation/feedback

Conversation

@shagun-singh-inkeep
Copy link
Copy Markdown
Collaborator

No description provided.

@vercel
Copy link
Copy Markdown

vercel bot commented Mar 31, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
agents-api Ready Ready Preview, Comment Apr 3, 2026 2:27pm
agents-manage-ui Ready Ready Preview, Comment Apr 3, 2026 2:27pm
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
agents-docs Skipped Skipped Apr 3, 2026 2:27pm

Request Review

@changeset-bot
Copy link
Copy Markdown

changeset-bot bot commented Mar 31, 2026

🦋 Changeset detected

Latest commit: 3342f58

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 10 packages
Name Type
@inkeep/agents-core Patch
@inkeep/agents-manage-ui Patch
@inkeep/agents-api Patch
@inkeep/agents-cli Patch
@inkeep/agents-sdk Patch
@inkeep/agents-work-apps Patch
@inkeep/ai-sdk-provider Patch
@inkeep/create-agents Patch
@inkeep/agents-email Patch
@inkeep/agents-mcp Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link
Copy Markdown
Contributor

@pullfrog pullfrog bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Medium priority — Solid implementation of the feedback feature end-to-end (schema → DAL → API → UI). A few issues to address: changeset message needs polish, the messageId FK on the feedback table creates a composite FK with nullable columns that needs careful handling during cascade, the CRUD test coverage is thin (missing update and get-by-id tests), and the feedback.ts API client duplicates the FeedbackListResponse type that already exists in agents-core.

Pullfrog  | Fix all ➔Fix 👍s ➔View workflow run | Using Claude Opus𝕏

"@inkeep/agents-api": patch
---

implementing feedback
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changeset message should be sentence-case with an action verb and describe the user-facing change, per AGENTS.md conventions.

Suggestion: Add feedback CRUD API, database table, and Manage UI for collecting user feedback on conversations and messages

total: number;
pages: number;
};
}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FeedbackListResponse is manually redefined here but already exists as a Zod schema in @inkeep/agents-core (FeedbackListResponse at validation/schemas.ts:2731). Consider importing and inferring the type from the canonical schema to avoid the two definitions drifting apart.

expect(listBody.pagination.total).toBe(0);
expect(listBody.data).toEqual([]);
});
});
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per the route-handler-authoring skill, CRUD tests must cover round-trip field persistence for all schema fields. This file is missing:

  1. A get-by-id test (GET /feedback/{id}) to verify the read path.
  2. An update test (PATCH /feedback/{id}) to verify partial updates persist — at minimum changing type and details.
  3. A filter-by-type test to exercise the type query parameter.

These gaps mean regressions in update or read-by-id handlers would go undetected.

query: PaginationQueryParamsSchema.extend({
conversationId: z.string().optional().describe('Optionally filter by conversation ID'),
messageId: z.string().optional().describe('Optionally filter by message ID'),
type: z.enum(['positive', 'negative']).optional().describe('Optionally filter by feedback type'),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The type query parameter is inlined directly in the route definition rather than being part of a reusable schema. This is fine for now but will be inconsistent if other routes need the same 'positive' | 'negative' enum — consider extracting to a shared FeedbackTypeSchema or at least referencing the enum from the FeedbackInsertSchema.

feedback: z
.string()
.min(1, 'Please provide details about what could have been better.')
.min(1, 'Please provide details.')
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The feedback field has a min(1) constraint requiring details text, but the API schema (FeedbackApiInsertSchema) allows details to be null/undefined. This means the dialog enforces stricter validation than the API — positive feedback always requires a text explanation, which may discourage quick thumbs-up interactions. Consider making the details field optional for positive feedback.

return;
}

const feedbackId = `feedback_${globalThis.crypto?.randomUUID?.() ?? Math.random().toString(36).slice(2)}`;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using Math.random().toString(36).slice(2) as a fallback for crypto.randomUUID() can produce collisions at scale (only ~31 bits of entropy). The API already accepts id as optional and generates one server-side via generateId() — consider just omitting id from the client request and letting the server assign it.

feedback: items,
total:
typeof total[0]?.count === 'string'
? Number.parseInt(total[0].count, 10)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The total coercion on line 85-87 handles the case where count is returned as a string (some PG drivers do this for bigint). However, parseInt with radix 10 on an already-number value returns the same number, so the ternary is correct. Just noting this is consistent with the codebase pattern — looks good.

* API Client for Feedback Operations
*/

'use server';
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is marked 'use server' but it's an API client module, not a Server Action. The 'use server' directive turns every exported function into a Server Action callable from the client. For a pure API client, this is unnecessary and could expose deleteFeedback and fetchFeedback as Server Actions — which means they'd be invocable directly from the browser, bypassing any Server Action-level guards in the actions/feedback.ts layer. Remove the 'use server' directive; the actions/feedback.ts wrappers already have it.

@vercel vercel bot temporarily deployed to Preview – agents-docs March 31, 2026 21:37 Inactive
@pullfrog
Copy link
Copy Markdown
Contributor

pullfrog bot commented Mar 31, 2026

TL;DR — Adds a full-stack feedback system that lets users submit thumbs-up/thumbs-down feedback on conversations and individual messages. This includes a new feedback table in the runtime database, CRUD API routes under the manage domain, a dedicated feedback page in the UI with filtering and pagination, and the ability to leave feedback directly from the playground chat widget and conversation trace timeline.

Key changes

  • New feedback runtime table — Drizzle schema + migration for a tenant/project-scoped feedback table with foreign keys to conversations and messages, plus indexes on both.
  • Full CRUD API routes — List, get, create, update, and delete endpoints mounted at /manage/tenants/:tenantId/projects/:projectId/feedback with project-level permission guards.
  • Data access layer — New feedback.ts DAL module with paginated listing, type filtering, and standard CRUD operations against the runtime database.
  • Zod validation schemasFeedbackSelectSchema, FeedbackInsertSchema, FeedbackUpdateSchema, and their API/response wrappers added to core validation.
  • Feedback page + table component — New /feedback page in the manage UI with a sortable, filterable table that supports conversation ID search, positive/negative type toggle, pagination, and inline delete.
  • Redesigned feedback dialog — Replaces the copilot-only "Improve with AI" flow with a persistent feedback form supporting sentiment selection (thumbs up/down), message vs. conversation scope, and server-side persistence; negative feedback still optionally triggers the copilot flow.
  • Trace timeline integration — Activities now carry messageId from OTEL spans, enabling a "Leave Feedback" button in the detail pane and message-level highlighting when navigating from the feedback table.
  • message.id OTEL attribute propagation — User and assistant message IDs are now set as span attributes in chat, data-stream, and execution handler routes, fixing a bug where conversationId was incorrectly used as the message ID in span events.
  • Sidebar navigation — "Feedback" entry added to the project sidebar.

Summary | 34 files | 3 commits | base: mainimplementation/feedback


Runtime database schema and migration

Before: No feedback storage existed.
After: A feedback table with composite PK (tenant_id, project_id, id), FK constraints cascading from conversations and messages, and indexes on conversation_id and message_id.

The table is defined in Drizzle with projectScoped + timestamps mixins, typed type column (positive | negative), optional message_id, and required conversation_id. Relations to conversations and messages are declared bidirectionally.

runtime-schema.ts · 0030_tough_zeigeist.sql · schemas.ts · entities.ts


API routes and data access layer

Before: No feedback endpoints existed.
After: Five OpenAPI routes (list, get by ID, create, update, delete) mounted under /manage/tenants/:tenantId/projects/:projectId/feedback, all protected by requireProjectPermission.

The DAL module provides listFeedback with optional conversationId, messageId, and type filters, parallelized count + data queries, and standard CRUD functions. All operations are tenant/project-scoped.

feedback.ts (routes) · feedback.ts (DAL) · feedback.test.ts


Feedback page and table in the manage UI

Before: No dedicated feedback view existed.
After: A new /feedback page with a paginated table showing timestamp, sentiment badge, linked conversation/message, truncated details, and delete action; filterable by conversation ID (debounced search) and type (positive/negative toggle).

The sidebar gains a "Feedback" entry with a MessageSquare icon. Server actions (createFeedbackAction, deleteFeedbackAction) handle API calls with revalidatePath for cache busting. An API client module provides typed createFeedback, fetchFeedback, and deleteFeedback functions.

feedback/page.tsx · feedback-table.tsx · delete-feedback-confirmation.tsx · app-sidebar.tsx · feedback.ts (actions) · feedback.ts (API client)


Redesigned feedback dialog with persistence

Before: The feedback dialog was a copilot-only "Fix with Copilot" flow that forwarded text to the copilot without persisting it. The "Improve with AI" message action was gated behind isCopilotConfigured.
After: The dialog now persists feedback to the API with sentiment (thumbs up/down) and scope (message vs. conversation) selection. The message action is always visible as "Leave Feedback". Negative feedback optionally still triggers the copilot flow via onNegativeFeedbackSubmit.

feedback-dialog.tsx · chat-widget.tsx


Trace timeline: message.id propagation and feedback integration

Before: The message.id span event attribute was incorrectly set to conversationId in chat routes; OTEL spans for assistant responses did not carry message.id; timeline activities had no messageId field.
After: message.id is correctly set as a span attribute for both user and assistant messages across chat.ts, chatDataStream.ts, and executionHandler.ts. The ActivityItem type gains an optional messageId. The conversation detail page supports ?messageId= query param for scroll-to-highlight, and a "Leave Feedback" button appears in the activity detail pane.

executionHandler.ts · chat.ts · chatDataStream.ts · otel-attributes.ts · timeline-wrapper.tsx · timeline-item.tsx · render-panel-content.tsx · conversationId/page.tsx

Pullfrog  | View workflow run | Triggered by Pullfrog | Using Claude Opus𝕏

@github-actions github-actions bot deleted a comment from claude bot Mar 31, 2026
@pullfrog
Copy link
Copy Markdown
Contributor

pullfrog bot commented Mar 31, 2026

TL;DR — Adds a full-stack feedback system that lets users submit positive/negative feedback on conversations and individual messages. This introduces a new feedback table in the runtime database, CRUD API routes under the manage domain, a dedicated Feedback page in the management UI, and inline "Leave Feedback" actions in the trace timeline and playground chat widget.

Key changes

  • New feedback runtime database table and migration — Stores per-conversation and per-message feedback with type (positive/negative) and free-text details, scoped by tenant and project.
  • Full CRUD API routes for feedback — List, get, create, update, and delete endpoints mounted at /manage/.../feedback with createProtectedRoute and requireProjectPermission, backed by a dedicated DAL module.
  • message.id propagated through OTEL spans — Execution handler and chat routes now set message.id on spans so the trace timeline can link activities to specific messages.
  • Feedback page and table in management UI — New sidebar entry, server-rendered page with paginated table, type/conversation filters, and inline delete.
  • Revamped feedback dialog — Replaces the "Fix with Copilot" flow with a proper feedback form supporting sentiment selection (thumbs up/down), message vs. conversation scope, and API persistence.
  • Message highlighting in trace timeline — Conversation detail page reads a messageId query param to visually highlight and auto-scroll to the matching timeline activity.
  • Leave Feedback button in trace detail panel — AI assistant message activities now surface a "Leave Feedback" action that opens the feedback dialog pre-scoped to that message.

Summary | 34 files | 1 commit | base: mainimplementation/feedback


New feedback table with foreign keys to conversations and messages

Before: No mechanism to store user feedback on agent responses.
After: A feedback table in the runtime database with columns for conversation_id, message_id, type (positive/negative), and details, plus cascade-delete foreign keys and composite indexes.

The table uses projectScoped and timestamps mixins for consistency with the existing schema. A Drizzle migration (0030_tough_zeigeist.sql) creates the table, foreign keys, and indexes. Drizzle relations link feedback back to conversations and messages.

runtime-schema.ts · 0030_tough_zeigeist.sql · schemas.ts · entities.ts


CRUD API routes and data access layer for feedback

Before: No feedback endpoints existed.
After: Five OpenAPI-registered routes (list, get, create, update, delete) under /manage/.../feedback with createProtectedRoute and requireProjectPermission, backed by a dedicated DAL module.

The DAL in packages/agents-core/src/data-access/runtime/feedback.ts provides getFeedbackById, listFeedback, listFeedbackByConversation, createFeedback, updateFeedback, and deleteFeedback — all tenant+project scoped. The list endpoint supports filtering by conversationId, messageId, and type with standard pagination. Validation uses FeedbackApiInsertSchema / FeedbackApiUpdateSchema derived from the Drizzle schema, and responses use FeedbackResponse / FeedbackListResponse Zod wrappers. Integration tests cover the create-list and create-delete round trips.

feedback.ts (routes) · feedback.ts (DAL) · index.ts (manage) · data-access/index.ts · feedback.test.ts (API)


message.id emitted on OTEL spans for trace-level feedback linking

Before: The message.id span attribute was either missing or incorrectly set to conversationId in chat routes.
After: Both chat.ts and chatDataStream.ts set message.id to the actual userMessageId on the message span, and executionHandler.ts sets it on the AI response span before persisting.

This allows the trace conversation route to extract message.id from span attributes and include it in ActivityItem, enabling the UI to highlight and link specific messages to feedback entries.

executionHandler.ts · chat.ts · chatDataStream.ts · otel-attributes.ts · conversations/route.ts


Feedback page and table in the management UI

Before: No dedicated UI for browsing feedback.
After: A new /feedback page accessible from the sidebar, with a filterable, paginated table showing type badges, conversation links, truncated details, and per-row delete.

The page is server-rendered via fetchFeedback and passes data to the client-side FeedbackTable component. Filters for sentiment type and conversation ID are synced to URL search params with debounced input. A DeleteFeedbackConfirmation alert dialog handles deletion with server action revalidation.

feedback/page.tsx · feedback-table.tsx · delete-feedback-confirmation.tsx · app-sidebar.tsx · theme.ts · api/feedback.ts · actions/feedback.ts · feedback.test.ts (actions)


Revamped feedback dialog with sentiment, scope, and API persistence

Before: The feedback dialog acted as a "Fix with Copilot" shortcut — it forwarded user text to the copilot chat without persisting feedback.
After: The dialog captures sentiment (thumbs up/down), scope (message vs. conversation), and details text, then persists via createFeedbackAction. Negative feedback optionally triggers the copilot flow via onNegativeFeedbackSubmit.

The playground ChatWidget now always shows a "Leave Feedback" message action (no longer gated on copilot being configured) and passes tenantId/projectId to the dialog. The copilot forwarding logic moved from inside the dialog to the onNegativeFeedbackSubmit callback in the widget.

feedback-dialog.tsx · chat-widget.tsx


Message highlighting and inline feedback in the trace timeline

Before: The conversation detail page had no way to highlight a specific message or leave feedback from the timeline.
After: A messageId query parameter highlights and auto-scrolls to the matching timeline activity. AI assistant message panels show a "Leave Feedback" button that opens the feedback dialog scoped to that message.

The ActivityItem type gains a messageId field. TimelineItem applies a ring highlight style when isHighlighted is true. TimelineWrapper accepts highlightMessageId and onLeaveFeedback props, threading them through to HierarchicalTimeline and renderPanelContent. The conversation detail page reads messageId from search params and wires up the feedback dialog.

conversations/[conversationId]/page.tsx · timeline-wrapper.tsx · timeline-item.tsx · hierarchical-timeline.tsx · render-panel-content.tsx · types.ts

Pullfrog  | View workflow run | Triggered by Pullfrog | Using Claude Opus𝕏

@pullfrog
Copy link
Copy Markdown
Contributor

pullfrog bot commented Mar 31, 2026

TL;DR — Adds a full-stack feedback system that lets users leave thumbs-up/thumbs-down feedback on individual messages or entire conversations. Feedback is persisted to a new feedback runtime table, exposed via CRUD API routes on the manage domain and a create endpoint on the run domain (with conversation ownership validation), and surfaced in the manage UI with a dedicated Feedback page, inline thumbs-up/thumbs-down actions in the chat widget and trace timeline, deep-links from the feedback table into the trace timeline, and a "View Feedback" link on the conversation detail page.

Key changes

  • Add feedback runtime database table and migration — New feedback table with FK references to conversations and messages, indexed on conversation and message IDs, with cascade deletes.
  • Add feedback data access layer with date range filtering — CRUD functions (listFeedback, createFeedback, updateFeedback, deleteFeedback) in agents-core with tenant/project scoping, pagination, a 100-row cap, and optional startDate/endDate query filters.
  • Add feedback CRUD API routes on the manage domain — Full REST API under /feedback in the manage domain with createProtectedRoute authorization (view for reads, edit for writes), including startDate and endDate query parameters for date range filtering.
  • Add end-user feedback submission route on the run domain — A create endpoint at /run/v1/feedback gated by inheritedRunApiKeyAuth, requiring endUserId and validating conversation ownership before persisting feedback.
  • Add integration tests for feedback routes — Six test cases covering create+list, get-by-id, update via PATCH, filter by type, conversation-level feedback without messageId, and delete.
  • Add Zod validation schemas and typesFeedbackSelectSchema, FeedbackInsertSchema, FeedbackUpdateSchema, and their API/response wrappers; FeedbackApiUpdateSchema omits immutable fields (conversationId, messageId, id).
  • Replace single "Leave Feedback" button with inline thumbs-up/thumbs-down actions — The chat widget and trace timeline panel now show separate thumbs-up and thumbs-down icon buttons that pre-select the feedback type in the dialog.
  • Simplify FeedbackDialog by removing scope selection — The dialog accepts initialType and onSubmitSuccess props, resets on open, and persists feedback server-side via createFeedbackAction. Scope is implicitly determined by whether a messageId is provided.
  • Add FeedbackTable with date picker, type filters, and delete support — New table with All/Positive/Negative button group (with count badges), a DatePickerWithPresets for date range filtering, right-aligned conversation links with ArrowUpRight icons, inline delete with confirmation dialog, and pagination controls.
  • Add "View Feedback" link on conversation detail page — When feedback exists for a conversation, a link appears in the conversation detail header navigating to the feedback table filtered by conversationId. Uses a hasConversationFeedbackAction that queries for existing feedback on page load and updates after submission.
  • Add Feedback sidebar navigation entry — New "Feedback" item with MessageSquare icon in the project sidebar nav.
  • Add message highlighting in trace timeline — Feedback table links include ?messageId= which highlights and auto-scrolls to the relevant message in the trace timeline.
  • Fix message.id OTEL attribute bug — Corrected message.id in span events from conversationId to the actual userMessageId in both chat.ts and chatDataStream.ts; added message.id as a span attribute on the execution handler.
  • Add server actions and API client for feedbackcreateFeedbackAction, hasConversationFeedbackAction, and deleteFeedbackAction server actions backed by a typed API client with FeedbackListResponse derived from FeedbackApiSelectSchema.
  • Clean up unused props and imports — Remove unused triggerClassName prop from DatePickerWithPresets, unused FeedbackDialog import from conversation page, and the onNegativeFeedbackSubmit copilot-forwarding callback from the chat widget.

Summary | 37 files | 23 commits | base: mainimplementation/feedback


Runtime feedback table and data access layer

Before: No mechanism to persist user feedback on agent responses.
After: A feedback table in the runtime database stores positive/negative feedback scoped to a conversation or specific message, with full CRUD data access functions and date range filtering.

The table uses the standard projectScoped columns and timestamps mixin. Foreign keys reference conversations and messages with ON DELETE CASCADE. The DAL uses the shared projectScopedWhere helper for tenant/project filtering, parallelizes count and data queries with Promise.all, caps pagination at 100 rows, and supports startDate/endDate parameters that apply gte/lte conditions on createdAt.

runtime-schema.ts · feedback.ts (DAL) · 0031_clumsy_prism.sql · schemas.ts


Feedback CRUD API on manage and run domains

Before: No API surface for feedback.
After: Five OpenAPI routes on the manage domain (list, get, create, update, delete) gated by requireProjectPermission with startDate/endDate query filters, plus a create-only endpoint on the run domain gated by inheritedRunApiKeyAuth with mandatory endUserId and conversation ownership validation.

Manage routes follow existing CRUD conventions under /manage/tenants/:tenantId/projects/:projectId/feedback. The list endpoint accepts optional startDate and endDate query parameters (YYYY-MM-DD format) for date range filtering. The run route lives at /run/v1/feedback and uses a requireEndUserId guard that extracts the end-user identity from executionContext.metadata. Before creating feedback, it fetches the conversation via getConversation and verifies that conversation.userId matches endUserId — returning a not_found error otherwise. The FeedbackApiUpdateSchema omits immutable fields (conversationId, messageId, id) to prevent mutation of feedback identity. The run feedback path is registered as a lightweight route in createApp.ts to skip the project config middleware prefetch. Six integration tests verify field persistence, type filtering, conversation-level feedback, and tenant/project scope stripping.

feedback.ts (manage routes) · feedback.ts (run routes) · feedback.test.ts


Inline thumbs-up/thumbs-down feedback actions

Before: A single "Leave Feedback" button opened a dialog defaulting to negative sentiment.
After: Separate thumbs-up and thumbs-down icon buttons in both the chat widget and the trace timeline panel pre-select the feedback type before opening the dialog.

The FeedbackDialog accepts initialType and onSubmitSuccess props and resets its form state on open. The scope (message vs. conversation) is implicitly determined by whether a messageId is provided — the previous explicit scope radio group was removed in favor of this simpler approach. Feedback is persisted server-side via createFeedbackAction. The conversation detail page's top-level "Leave Feedback" button was removed in favor of inline per-message actions.

chat-widget.tsx · render-panel-content.tsx · feedback-dialog.tsx · conversation page.tsx


Feedback table with date range filtering and delete support

Before: No dedicated feedback management page existed.
After: A new /feedback page with an All/Positive/Negative button group (with count badges), a DatePickerWithPresets for date range filtering, right-aligned ArrowUpRight icon buttons linking to conversations (with ?messageId= for deep-linking), inline delete with a confirmation dialog, and paginated results.

The FeedbackTable passes startDate/endDate through URL search params to the server component, which forwards them to the fetchFeedback API call. Filter state flows through the page's searchParams for SSR. A DeleteFeedbackConfirmation dialog handles removal with deleteFeedbackAction and triggers router.refresh() on success.

feedback-table.tsx · page.tsx (feedback) · delete-feedback-confirmation.tsx · feedback.ts (API client)


"View Feedback" link on conversation detail page

Before: No way to navigate from a conversation to its feedback.
After: When feedback exists for a conversation, a "View Feedback" link appears in the conversation detail header, navigating to the feedback table pre-filtered by conversationId.

On page load, hasConversationFeedbackAction fires in parallel with the trace and events fetches via Promise.allSettled. It queries the feedback list endpoint with limit: 1 and sets a hasFeedback boolean. When the user submits feedback via the dialog, the onSubmitSuccess callback flips the flag without a re-fetch, making the link appear immediately.

conversation page.tsx · feedback.ts (actions) · feedback-dialog.tsx


Message highlighting in trace timeline

Before: No way to deep-link to a specific message in the trace timeline.
After: Timeline items expose data-message-id attributes; a highlightMessageId prop triggers a ring highlight and smooth scroll-into-view.

The ActivityItem type gains a messageId field populated from the message.id OTEL attribute. TimelineItem renders a visual ring when isHighlighted is true. The conversation detail page reads ?messageId from search params and passes it through the timeline component tree.

timeline-wrapper.tsx · timeline-item.tsx · types.ts


Fix message.id OTEL attribute bug

Before: The message.id span event attribute was incorrectly set to conversationId.
After: Corrected to userMessageId in both chat.ts and chatDataStream.ts; also added message.id as a top-level span attribute in the execution handler so AI response spans carry the response message ID.

chat.ts · chatDataStream.ts · executionHandler.ts

Pullfrog  | View workflow run | Triggered by Pullfrog | Using Claude Opus𝕏

Copy link
Copy Markdown
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(8) Total Issues | Risk: Medium

🟠⚠️ Major (3) 🟠⚠️

Inline Comments:

  • 🟠 Major: feedback-dialog.tsx:38 Feedback text required for positive sentiment creates unnecessary friction
  • 🟠 Major: feedback-table.tsx:186-194 Search input missing accessible label (WCAG 2.1 failure)
  • 🟠 Major: feedback.test.ts:264 Missing 404 error handling tests for GET, PATCH, DELETE

🟡 Minor (5) 🟡

Inline Comments:

  • 🟡 Minor: feedback-table.tsx:158-181 Filter toggle buttons missing aria-pressed attribute
  • 🟡 Minor: feedback-table.tsx:196-207 Clear filter button missing accessible label
  • 🟡 Minor: chat-widget.tsx:259-262 Icon mismatch - sparkles icon may confuse users expecting AI flow

🟡 1) feedback-dialog.tsx:102-110 Negative feedback copilot behavior inconsistent across contexts

Issue: When negative feedback is submitted in the playground, the dialog calls onNegativeFeedbackSubmit to open the copilot and auto-submit the feedback text. In the conversation detail page (traces), this callback is not passed, so negative feedback just saves without the copilot step.

Why: Users who learn the workflow in playground ("negative feedback opens copilot") may be confused when the same action in traces just saves feedback. The dialog shows the same "Submit feedback" button in both cases, but outcomes differ.

Fix: Either make the copilot integration explicit in the UI (e.g., checkbox "Open copilot to help fix" only shown when available), or ensure the callback is available in all contexts, or document the different behavior.

Refs: conversation/page.tsx:568-575

🟡 2) feedback.ts (DAL):14-20 DAL does not use projectScopedWhere helper

Issue: The feedback DAL manually constructs scope conditions using explicit eq(feedback.tenantId, ...) calls instead of using the projectScopedWhere helper that all other runtime DAL modules use.

Why: All peer runtime DAL modules (messages.ts, conversations.ts, tasks.ts) use projectScopedWhere for consistency. This inconsistency makes the codebase harder to maintain and could lead to subtle scoping bugs if the helper is updated.

Fix: Import and use projectScopedWhere from ../manage/scope-helpers:

where: and(projectScopedWhere(feedback, params.scopes), eq(feedback.id, params.feedbackId))

Refs: messages.ts:11-17

💭 Consider (3) 💭

💭 1) feedback.test.ts:196 Missing test for conversationId filter
Issue: Only type filter is tested. The conversationId and messageId filters are untested.
Fix: Add test that creates feedback for two conversations, verifies filter returns only expected entries.

💭 2) feedback-dialog.tsx:53-57 Default sentiment is 'negative'
Issue: Defaulting to negative may bias feedback collection. Consider whether this serves product goals or creates unintentional bias.
Fix: Consider no default (require user choice) or context-dependent defaults.

💭 3) feedback-table.tsx:193 Filter placeholder uses camelCase developer terminology
Issue: Placeholder says "Filter by conversationId" but column header says "Conversation".
Fix: Change to "Filter by conversation ID" for consistency.

🧹 While You're Here (0) 🧹

No pre-existing issues identified.


💡 APPROVE WITH SUGGESTIONS

Summary: Solid full-stack implementation of the feedback system. The schema, DAL, API routes, and UI components follow established patterns well. However, there are 3 Major issues that should be addressed:

  1. Accessibility gaps in the feedback table (missing aria-labels, aria-pressed)
  2. Positive feedback requiring text creates UX friction that may reduce positive signal collection
  3. Missing 404 error handling tests leave API contract gaps untested

The changeset message has been updated to follow conventions (per prior Pullfrog review). The OTEL message.id bug fix is a nice catch and correctly implemented.

Discarded (5)
Location Issue Reason Discarded
feedback-dialog.tsx:137-139 Redundant sr-only text alongside visible button label The sr-only span provides additional context and is a valid accessibility pattern when the icon adds meaning beyond the text.
feedback-table.tsx:63-64 Potential duplicate state synchronization with URL params This is a standard pattern for debounced inputs - local state for input value, URL for source of truth. Works correctly.
timeline-item.tsx:246 String concatenation for className This pattern is used elsewhere in the codebase and works correctly. Minor style preference.
feedback.ts (DAL):91-103 createFeedback signature differs from peer functions While true, the current signature is functional and changing it would require API route changes. Low-impact consistency issue.
FeedbackTable uses manual Table Doesn't use DataTable component Both patterns exist in the codebase (DataTable and manual Table). This follows the manual pattern which is valid for simpler tables.
Reviewers (6)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
pr-review-frontend 9 0 0 1 4 0 4
pr-review-product 5 1 2 0 1 0 1
pr-review-tests 3 0 1 0 1 0 1
pr-review-consistency 4 1 0 0 0 0 3
pr-review-breaking-changes 0 0 0 0 0 0 0
pr-review-standards 0 0 0 0 0 0 0
Total 21 2 3 1 6 0 9

Note: pr-review-breaking-changes confirmed migration is additive and safe. No schema breaking changes.

.string()
.min(1, 'Please provide details about what could have been better.')
.min(1, 'Please provide details.')
.max(1000, 'Feedback must be less than 1000 characters'),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 MAJOR: Feedback text required for positive sentiment creates friction

Issue: The .min(1) validation requires users to type at least one character even for positive (thumbs up) feedback. Most feedback systems allow one-click positive ratings.

Why: Requiring detailed text for a simple "this was good" signal reduces positive feedback submission rates and makes quick positive signals feel heavyweight.

Fix: Make feedback text optional for positive sentiment:

Suggested change
.max(1000, 'Feedback must be less than 1000 characters'),
feedback: z
.string()
.max(1000, 'Feedback must be less than 1000 characters')
.optional(),

Then add conditional validation in onSubmit if type === 'negative' requires details.

Comment on lines +186 to +194
<Input
value={conversationId}
onChange={(e) => {
const v = e.target.value;
setConversationId(v);
debouncedConversationFilter(v);
}}
placeholder="Filter by conversationId"
className="pl-8 pr-8"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 MAJOR: Search input missing accessible label (WCAG 2.1 failure)

Issue: The search Input has no associated label or aria-label. Screen reader users won't understand the purpose of this field. Placeholders are not reliable accessible names.

Why: WCAG 2.1 Level A failure (1.3.1 Info and Relationships, 4.1.2 Name, Role, Value).

Fix:

Suggested change
<Input
value={conversationId}
onChange={(e) => {
const v = e.target.value;
setConversationId(v);
debouncedConversationFilter(v);
}}
placeholder="Filter by conversationId"
className="pl-8 pr-8"
<div className="relative w-full md:w-[260px]">
<Search className="absolute left-2 top-1/2 -translate-y-1/2 h-4 w-4 text-gray-400 dark:text-white/40" aria-hidden="true" />
<Input
aria-label="Filter by conversation ID"
value={conversationId}
onChange={(e) => {
const v = e.target.value;
setConversationId(v);
debouncedConversationFilter(v);
}}
placeholder="Filter by conversationId"
className="pl-8 pr-8"
/>

Refs:

Comment on lines +158 to +181
<Button
variant={typeFilter === 'positive' ? 'default' : 'outline'}
size="sm"
onClick={() => {
const next = typeFilter === 'positive' ? undefined : ('positive' as const);
setTypeFilter(next);
updateQuery({ type: next ?? '', page: 1 });
}}
>
<ThumbsUp className="h-3.5 w-3.5 mr-1" />
Positive
</Button>
<Button
variant={typeFilter === 'negative' ? 'default' : 'outline'}
size="sm"
onClick={() => {
const next = typeFilter === 'negative' ? undefined : ('negative' as const);
setTypeFilter(next);
updateQuery({ type: next ?? '', page: 1 });
}}
>
<ThumbsDown className="h-3.5 w-3.5 mr-1" />
Negative
</Button>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Minor: Filter toggle buttons missing aria-pressed attribute

Issue: The Positive/Negative filter buttons act as toggles but don't expose their pressed state to assistive technology.

Why: Screen reader users won't know which filter is currently active.

Fix: Add aria-pressed to each toggle button:

<Button
  variant={typeFilter === 'positive' ? 'default' : 'outline'}
  size="sm"
  aria-pressed={typeFilter === 'positive'}
  onClick={() => { ... }}
>

Refs:

Comment on lines +196 to +207
{conversationId ? (
<Button
variant="ghost"
size="icon"
className="absolute right-0 top-0 h-9 w-9"
onClick={() => {
setConversationId('');
debouncedConversationFilter('');
}}
>
<X className="h-4 w-4" />
</Button>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Minor: Clear filter button missing accessible label

Issue: The X icon button lacks an aria-label. The delete button on line 267 correctly has aria-label="Delete feedback", but this clear button doesn't follow the same pattern.

Why: Screen reader users will hear only "button" with no indication of what it does.

Fix:

Suggested change
{conversationId ? (
<Button
variant="ghost"
size="icon"
className="absolute right-0 top-0 h-9 w-9"
onClick={() => {
setConversationId('');
debouncedConversationFilter('');
}}
>
<X className="h-4 w-4" />
</Button>
{conversationId ? (
<Button
variant="ghost"
size="icon"
className="absolute right-0 top-0 h-9 w-9"
aria-label="Clear filter"
onClick={() => {
setConversationId('');
debouncedConversationFilter('');
}}
>
<X className="h-4 w-4" />
</Button>

Comment on lines +259 to +262
messageActions: [
{
label: 'Leave Feedback',
icon: { builtIn: 'LuSparkles' },
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Minor: Icon mismatch - sparkles icon may confuse users

Issue: The message action uses LuSparkles icon with 'Leave Feedback' label, but the feedback dialog uses ThumbsUp/ThumbsDown and the timeline uses MessageSquare. The sparkles icon was previously associated with 'Improve with AI'.

Why: Users may expect sparkles to trigger an AI-assisted flow (its previous purpose), not a feedback dialog. Icon vocabulary should be consistent across surfaces.

Fix: Change to LuMessageSquare to match the sidebar and timeline:

Suggested change
messageActions: [
{
label: 'Leave Feedback',
icon: { builtIn: 'LuSparkles' },
messageActions: [
{
label: 'Leave Feedback',
icon: { builtIn: 'LuMessageSquare' },

Copy link
Copy Markdown
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(0) Total Issues | Risk: Low

Delta Review Summary

This is a re-review scoped to changes since commit c3fca534e. The delta consists of 3 files with targeted fixes that address previously raised issues.

File Change Assessment
feedback-dialog.tsx Schema refactored with .refine() — positive feedback text now optional ✅ Addresses prior Major (feedback text required for positive)
feedback-table.tsx Added aria-pressed, aria-label, aria-hidden attributes ✅ Addresses prior Major (WCAG) + Minor (accessibility) issues
feedback.ts (DAL) Refactored to use projectScopedWhere helper ✅ Addresses prior Minor (consistency with peer DAL modules)

Changes Validated

1. Feedback Dialog Schema (feedback-dialog.tsx:32-41)

The validation schema now correctly:

  • Makes feedback field optional at the schema level (.optional())
  • Uses .refine() to conditionally require text only for negative feedback
  • Properly handles edge cases (empty strings are falsy, so data.feedback && data.feedback.length > 0 is correct)

2. Accessibility Improvements (feedback-table.tsx)

All accessibility issues from prior review are addressed:

  • Filter toggle buttons now have aria-pressed (lines 161, 174)
  • Search input has aria-label="Filter by conversation ID" (line 189)
  • Clear filter button has aria-label="Clear filter" (line 204)
  • Search icon has aria-hidden="true" (line 187)

3. DAL Consistency (feedback.ts)

The DAL now uses projectScopedWhere(feedback, params.scopes) helper consistently across all functions (getFeedbackById, listFeedback, updateFeedback, deleteFeedback), matching the pattern used by peer runtime DAL modules.

🕐 Pending Recommendations (5)

Prior issues from the initial review that remain applicable (not in delta scope):


✅ APPROVE

Summary: The delta changes successfully address the Major accessibility issues and the positive-feedback-friction concern from the prior review. The DAL refactor improves codebase consistency. No new issues introduced. The remaining Pending Recommendations are from unchanged code and can be addressed in follow-up work.

Reviewers (1)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
orchestrator 3 0 0 0 0 5 0
Total 3 0 0 0 0 5 0

Note: Delta review — sub-reviewers not dispatched for this scope. Prior findings evaluated against delta changes.

Copy link
Copy Markdown
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(0) Total Issues | Risk: Low

Delta Review Summary

This is a re-review scoped to changes since commit 48e6b3e0e. The delta consists of 4 files with 1 commit (ddfc0b467 added filter to trace page).

File Change Assessment
page.tsx (conversation detail) Added feedback existence check + "View Feedback" link + onSubmitSuccess callback ✅ Clean — improves UX by showing feedback link when applicable
feedback-dialog.tsx Added onSubmitSuccess prop ✅ Clean — enables immediate UI update after submission
feedback-table.tsx Layout alignment fixes (text-right, justify-end) ✅ Clean — minor CSS improvements
feedback.ts (actions) Added hasConversationFeedbackAction server action ✅ Clean — efficient existence check with proper error handling

Changes Validated

1. New Server Action (hasConversationFeedbackAction):

  • Correctly uses 'use server' directive
  • Efficient implementation with limit: 1 (only checks if at least one exists)
  • Safe error handling — returns false on any failure, which is appropriate for a "has feedback" display check
  • Reuses existing fetchFeedback API client

2. Conversation Detail Integration (page.tsx):

  • Uses Promise.allSettled to parallelize the feedback check with existing data fetches — no additional latency on the critical path
  • Properly handles the settled promise result with type-safe status check
  • Conditionally shows "View Feedback" link only when feedback exists
  • Updates hasFeedback state via onSubmitSuccess after successful in-page feedback submission — good UX that avoids requiring a refresh

3. FeedbackDialog Enhancement:

  • New optional onSubmitSuccess callback is properly typed and called after successful submission
  • No breaking changes to existing usages

4. Table Layout Fixes:

  • Column width and alignment improvements for the action column
  • No functional changes

🕐 Pending Recommendations (1)

Prior issue that remains applicable (not in delta scope):

  • 🟡 feedback-table.tsx:148-155 'All' button missing aria-pressed for accessibility consistency — low priority, can be addressed in follow-up

✅ APPROVE

Summary: The delta changes are clean additions that improve the feedback feature's usability. The new hasConversationFeedbackAction server action efficiently checks for existing feedback, and the conversation detail page now shows a "View Feedback" link when applicable. The onSubmitSuccess callback enables immediate UI updates without requiring a page refresh. No new issues introduced.

Note: Unable to submit formal approval due to GitHub App permissions. This review recommends approval.

Discarded (0)

No findings discarded in this delta review.

Reviewers (1)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
orchestrator 4 0 0 0 0 1 0
Total 4 0 0 0 0 1 0

Note: Delta review — sub-reviewers not dispatched as delta scope is minimal (+35/-4 lines, UX polish only). Prior security and accessibility findings remain resolved.

Copy link
Copy Markdown
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(0) Total Issues | Risk: Low

Delta Review Summary

This is a re-review scoped to changes since commit ddfc0b467fb9. The delta consists of 2 files with 1 commit (a23fca9c4 lint).

File Change Assessment
feedback.test.ts Fixed relative import paths (lines 4-7) ✅ Clean — corrects path resolution from test location
chat-widget.tsx Prefixed unused param with underscore (setShowTraces_setShowTraces) ✅ Clean — standard lint practice for unused destructured props

No new issues introduced in the delta. Both changes are pure lint fixes that don't affect runtime behavior.

🕐 Pending Recommendations (1)

Prior issue that remains applicable (not in delta scope):

  • 🟡 feedback-table.tsx:148-155 'All' button missing aria-pressed for accessibility consistency — low priority, can be addressed in follow-up

✅ APPROVE

Summary: The delta changes are lint fixes only — correcting import paths in the test file and prefixing an unused prop with underscore in the chat widget. No functional changes, no new issues introduced. All prior Major security issues (IAM bypass, mass assignment, missing ownership validation) have been resolved in earlier commits. The feature is ready for merge.

The one remaining Minor issue (aria-pressed on the 'All' filter button) is documented as a follow-up item and does not block approval.

Note: Unable to submit formal approval due to GitHub App permissions. This review recommends approval.

Discarded (0)

No findings discarded in this delta review.

Reviewers (1)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
orchestrator 2 0 0 0 0 1 0
Total 2 0 0 0 0 1 0

Note: Delta review — sub-reviewers not dispatched as delta scope is minimal (lint fixes only, 2 files, -5/+5 lines). Prior security and accessibility findings remain resolved.

Copy link
Copy Markdown
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(0) Total Issues | Risk: Low

Delta Review Summary

This is a re-review scoped to changes since commit a23fca9c4. The delta consists of 1 file with 1 commit (d46333aaa fix tests).

File Change Assessment
feedback.test.ts Added vi.mock('next/cache', ...) for revalidatePath Clean fix — required mock for server action tests

No new issues introduced in the delta. The change adds a necessary mock for revalidatePath from next/cache, which is called by the createFeedbackAction and deleteFeedbackAction server actions. Without this mock, tests would fail because revalidatePath is a Next.js-specific function that doesn't exist in the test environment.

🕐 Pending Recommendations (1)

Prior issue that remains applicable (not in delta scope):

  • 🟡 feedback-table.tsx:148-155 'All' button missing aria-pressed for accessibility consistency — low priority, can be addressed in follow-up

✅ APPROVE

Summary: The delta change is a test infrastructure fix that adds the required next/cache mock for server action tests. This is a standard pattern for testing Next.js server actions and correctly fixes what was likely a test failure. All prior Major security issues (IAM bypass, mass assignment, ownership validation) have been resolved in earlier commits. The feedback feature is well-implemented and ready for merge.

The one remaining Minor accessibility issue (aria-pressed on the 'All' filter button) is documented as a follow-up item and does not block approval.

Note: Unable to submit formal approval due to GitHub App permissions. This review recommends approval.

Discarded (0)

No findings discarded in this delta review.

Reviewers (1)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
orchestrator 1 0 0 0 0 1 0
Total 1 0 0 0 0 1 0

Note: Delta review — sub-reviewers not dispatched as delta scope is minimal (test mock fix only, 1 file, +4 lines). Prior security and accessibility findings remain resolved.

Copy link
Copy Markdown
Contributor

@claude claude bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Review Summary

(0) Total Issues | Risk: Low

Delta Review Summary

This is a re-review scoped to changes since commit d46333aaa. The delta consists of 2 feedback-related files with cleanup/optimization changes plus a merge from main.

File Change Assessment
feedback-dialog.tsx Changed form.watch('type')useWatch({ control, name }) Clean — React Compiler optimization, prevents unnecessary re-renders
feedback-dialog.tsx Removed unused onNegativeFeedbackSubmit prop Clean — dead code removal
delete-feedback-confirmation.tsx Moved setIsDeleting(false) outside finally block Clean — fixes subtle bug where finally ran after early return

No new issues introduced in the delta. The changes are cleanup optimizations following the React Compiler migration merged from main.

🕐 Pending Recommendations (1)

Prior issue that remains applicable (not in delta scope):

  • 🟡 feedback-table.tsx:148-155 'All' button missing aria-pressed for accessibility consistency — low priority, can be addressed in follow-up

✅ APPROVE

Summary: The delta changes are clean optimizations that improve React performance and remove dead code. The useWatch change follows React Compiler best practices. The finally block fix prevents state being reset after early returns. All prior Major security issues (IAM bypass, mass assignment, ownership validation) were resolved in earlier commits and remain intact. The feedback feature is production-ready.

The one remaining Minor accessibility issue (aria-pressed on the 'All' filter button) is documented as a follow-up item and does not block approval.

Note: Unable to submit formal approval due to GitHub App permissions. This review recommends approval.

Discarded (0)

No findings discarded in this delta review.

Reviewers (1)
Reviewer Returned Main Findings Consider While You're Here Inline Comments Pending Recs Discarded
orchestrator 2 0 0 0 0 1 0
Total 2 0 0 0 0 1 0

Note: Delta review — sub-reviewers not dispatched as delta scope is minimal (cleanup/optimization changes in 2 feedback files + merge from main). Prior security and accessibility findings remain resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant