hhmx.de

Föderation EN Di 29.10.2024 01:50:42

@renchap poking at the new support in 4.3.0, and there's a bunch of neat data here. yay!

is there any value in keeping these repeated ZSCAN traces? they're the vast majority of our current event usage, and they don't seem to be terribly meaningful. usually (maybe exclusively) just a span that says ZSCAN.

safe to drop these? i don't think we'll lose any meaningful observability by doing so?

Medien: 2

Föderation EN Di 29.10.2024 02:15:12

@esk having used sidekiq before I’d say drop them and/or heavily sample them if you can. It should just be Sidekiq looking for jobs in sorted order. It’s something that would be nice to have a metric for to know if it’s starting to bog down workers or something, but a span for them all is a bit much IMO.

Also yay Honeycomb!!

Föderation EN Di 29.10.2024 09:44:53

@esk Are they part of a trace, and/or do they have a parent span?
I think they are Sidekiq workers polling Redis and trying to see if there are any jobs to process, in which case they can be filtered (but maybe not all ZSCANs?). If thats the case, we should probably filter them out directly in our instrumentation, or better sample them heavily.
For mastodon.social we only keep 1% of traces, except for errors where we keep 100% of them.

Föderation EN Di 29.10.2024 15:13:25

@renchap they seem to be the full trace - no parent/children that I can find (granted, i'm a big honeycomb noob).

we plan to turn on sampling for things here shortly to make the total number of traces less... overwhelming. :)

Föderation EN Di 29.10.2024 16:21:08

@esk So this can probably be dropped, and if you can open a PR on the Mastodon repo that either heavily sample or maybe drop those single calls, it will probably get merged as I dont really see how those would be useful.