Skip to content

.NET: [Bug]: AG-UI ASP.NET Core endpoint crashes on multimodal user messages (content array) #3729

@thoemmi

Description

@thoemmi

Description

Hi team,

I’m building an AG-UI chat app with a .NET server using MapAGUI(...). Text-only chat works fine, but as soon as I send a user message with an attached image, the request fails during JSON binding.

What I’m seeing

Server throws this when content is an array:
Microsoft.AspNetCore.Http.BadHttpRequestException: Failed to read parameter "RunAgentInput? input" from the request body as JSON.
---> System.Text.Json.JsonException: The JSON value could not be converted to System.String. Path: $.content

AG-UI clients can send multimodal user messages where content is an array (text + binary), see https://docs.ag-ui.com/sdk/js/core/types#usermessage
The current implementation appears to expect content to always be a plain string.
So the request fails before the agent even runs.

Code Sample

1. Set up AG-UI endpoint normally:
  
  builder.Services.AddAGUI();
  app.MapAGUI("/ag-ui", agent);
  

2. POST this to `/ag-ui`:
  
  {
    "threadId": "thread-1",
    "runId": "run-1",
    "messages": [
        "id": "m1",
        "role": "user",
        "content": [
          { "type": "text", "text": "What is in this image?" },
          {
            "type": "binary",
            "mimeType": "image/png",
            "filename": "pixel.png",
            "data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8Xw8AAoMBgP6x2JwAAAAASUVORK5CYII="
          }
        ]
      }
    ],
    "context": []
  }
  

3. Result: request fails with the exception below.

Error Messages / Stack Traces

fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1]
        An unhandled exception has occurred while executing the request.
        Microsoft.AspNetCore.Http.BadHttpRequestException: Failed to read parameter "RunAgentInput? input" from the request body as JSON.
         ---> System.Text.Json.JsonException: The JSON value could not be converted to System.String. Path: $.content | LineNumber: 0 | BytePositionInLine: 70.
         ---> System.InvalidOperationException: Cannot get the value of a token type 'StartArray' as a string.
           at System.Text.Json.ThrowHelper.ThrowInvalidOperationException_ExpectedString(JsonTokenType tokenType)
           at System.Text.Json.Utf8JsonReader.GetString()
           at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
           at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, T& value, JsonSerializerOptions options, ReadStack& state)
           --- End of inner exception stack trace ---
           at System.Text.Json.ThrowHelper.ReThrowWithPath(ReadStack& state, Utf8JsonReader& reader, Exception ex)
           at System.Text.Json.Serialization.JsonConverter`1.ReadCore(Utf8JsonReader& reader, T& value, JsonSerializerOptions options, ReadStack& state)
           at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Deserialize(Utf8JsonReader& reader, ReadStack& state)
           at System.Text.Json.JsonSerializer.Deserialize(JsonElement element, JsonTypeInfo jsonTypeInfo)
           at Microsoft.Agents.AI.Hosting.AGUI.AspNetCore.Shared.AGUIMessageJsonConverter.Read(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options)
           at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
           at System.Text.Json.Serialization.Metadata.JsonPropertyInfo`1.ReadJsonAndSetMember(Object obj, ReadStack& state, Utf8JsonReader& reader)
           at System.Text.Json.Serialization.Converters.ObjectDefaultConverter`1.OnTryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value)
           at System.Text.Json.Serialization.JsonConverter`1.TryRead(Utf8JsonReader& reader, Type typeToConvert, JsonSerializerOptions options, ReadStack& state, T& value, Boolean& isPopulatedValue)
           at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.ContinueDeserialize[TReadBufferState,TStream](TReadBufferState& bufferState, JsonReaderState& jsonReaderState, ReadStack& readStack, T& value)
           at Microsoft.AspNetCore.Http.HttpRequestJsonExtensions.ReadFromJsonAsync[TValue](HttpRequest request, JsonTypeInfo`1 jsonTypeInfo, CancellationToken cancellationToken)
           at Microsoft.AspNetCore.Http.HttpRequestJsonExtensions.ReadFromJsonAsync[TValue](HttpRequest request, JsonTypeInfo`1 jsonTypeInfo, CancellationToken cancellationToken)

Package Versions

Microsoft.Agents.AI.Hosting.AGUI.AspNetCore 1.0.0-preview.260205.1

.NET Version

10.0.102

Additional Context

No response

Metadata

Metadata

Assignees

Labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions