OpenAIChatClient streaming tool calls may be broken

[SteveSandersonMS]

Description

As of today, some app code that was working before is now failing with:

System.ArgumentNullException: Value cannot be null. (Parameter 'bytes')
2024-10-24T15:00:53          at System.ArgumentNullException.Throw(String paramName)
2024-10-24T15:00:53          at System.Text.StringBuilder.Append(Object value)
2024-10-24T15:00:53          at Microsoft.Extensions.AI.OpenAIChatClient.CompleteStreamingAsync(IList`1 chatMessages, ChatOptions options, CancellationToken cancellationToken)+MoveNext()
2024-10-24T15:00:53          at Microsoft.Extensions.AI.OpenAIChatClient.CompleteStreamingAsync(IList`1 chatMessages, ChatOptions options, CancellationToken cancellationToken)+MoveNext()
2024-10-24T15:00:53          at Microsoft.Extensions.AI.OpenAIChatClient.CompleteStreamingAsync(IList`1 chatMessages, ChatOptions options, CancellationToken cancellationToken)+System.Threading.Tasks.Sources.IValueTaskSource<System.Boolean>.GetResult()

this happens if and only if I try to do function calling with CompleteStreamingAsync. It seems that the FunctionArgumentsUpdate may now be a BinaryData instance that internally holds an empty _data value. This causes the exception.

Perhaps the underlying OpenAI/AzureOpenAI service has started returning data in a different form.

Or perhaps I've caused this in my own code by some hackery. But I've got a talk to do in a few hours so right now I'm just keeping track of this and putting in this workaround into CompleteStreamingAsync:

if (toolCallUpdate.FunctionArgumentsUpdate is not null
    /* workaround */ && toolCallUpdate.FunctionArgumentsUpdate.ToArray().Length > 0)
{
    _ = (existing.Arguments ??= new()).Append(toolCallUpdate.FunctionArgumentsUpdate);
}

Reproduction Steps

This is in eShopSupport.

If this doesn't repro with other streaming function calling, I'll come back to work out more repro details.

Expected behavior

No exception

Actual behavior

Exception

Regression?

No response

Known Workarounds

No response

Configuration

No response

Other information

No response

[SteveSandersonMS]

In general it seems you can't append an empty BinaryData to a StringBuilder:

existing.Arguments!.Append(new BinaryData([])); // Throws

To me this seems like either a bug in StringBuilder (why shouldn't it accept empty values to append?), or perhaps it was never legitimate to append a BinaryData to a StringBuilder in the first place.

[stephentoub]

Can you try explicitly referencing a recent version of System.Memory.Data from your app? I suspect this is a case where the Azure libs are referencing older versions, nothing else in the app is causing it to pull in a newer one, and it's thus missing bug fixes from the last several years.

[stephentoub]

e.g. this throws when referencing System.MemoryData 6.0.0

new BinaryData(ReadOnlyMemory<byte>.Empty).ToString();

and succeeds when changing it to 7.0.0.