Skip to content

Serialization error invalidates entire batch of events #180

Description

@tcollier

I recently fixed an event tracking-related bug in my Rails app and while the fix was squarely in my app, I bumped into a few surprises with the analytics-ruby gem that I think offer opportunities for improvement.

My bug was that a controller invoked .track with an event including a nested ActionController::Parameters property, which was not properly "permitted". The symptom of the bug was missing events in our downstream destinations. The fix was simple: call .permit on the parameters that go in the event payload.

The first surprise with analytics-ruby I encountered was that the on_error handler printed the error

Connection error: unable to convert unpermitted parameters to hash

The first part of the message ("Connection error") incorrectly led me to believe that the issue was in establishing a connection with Segment servers. This nudged my investigation towards HTTP-related issues and I eventually opened up a support ticket with Segment. We eventually realized the issue was a serialization error and were then able to correctly identify the problem and fix it. Had the error been labeled "Serialization error:" or something similar, finding the root of the bug would've taken much less time.

The second surprise was that not only did we lose the events with an unserializable property (the "bad events"), but we also lost an order of magnitude more unrelated "good" events. Any good event in the same Rails process as a bad event had a chance of not making it to our destinations. My unsubstantiated hunch is that if a bad event is enqueued with a .track call, then all events in the batch (good or bad) will fail to make it to Segment servers.

Finally, while I was aware of the batching functionality of analytics-ruby, I was still surprised that the serialization error occurred asynchronously in the worker thread, rather than synchronously in the call to .track. The latter would've made it easier to detect and handle.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions