Upipe
New struct uref member similar to priv
We have a need or desire for a new struct uref member similar to priv but big enough to store at least an IPv6 address in the future. The use for us would be the same as the packet info attributes added in #1021. That attribute works very well but it is surprisingly costly adding an extra 50% to the CPU requirements just for network reception. A thread almost dedicated to upipe_udp_source jumped from to about 25% usage when RXing a 93Mbit stream (4k video split into quadrants) and perf showed a lot of it was caused by the extra malloc to hold the opaque data in the udict.
This lead me to try using the priv member for just the IPv4 address which did work in most places but I discovered that upipe_rtp_feedback uses the member itself (correctly) and overwrites that value I had in it. It did remove the overhead of the new attribute so it was promising. It also showed the poor idea of using something supposed to be for internal pipe usage.
I am making this issue to hear the opinions of other contributors, particularly on the following points.
- Is this a good idea?
- Do other people want it, have a use for it?
- What should it be called?
- How big does it need to be?
My idea is that it would only be used for high throughput urefs such as those coming from a network socket, not to replace attributes on other less frequent urefs. For us it could be at least as big as struct sockaddr_storage which is 128 bytes on my desktop.
I am going to work on adding it at that size and call it "priv2" for now.
The basic idea was relatively simple. It can be seen in this commit https://github.com/JDarnley/upipe/commit/9c667b91391f7adb858ffa52186889825b894a82 It works, in a development branch, to pass source IP details through the pipeline for some downstream pipes to use.
If your extra CPU usage comes from umem allocations from the udict_inline implementation, perhaps it could be solved by adjusting the min_size parameter for udict_inline_mgr_alloc(). The default value (128) is too low for storing the existing attributes plus a sockaddr_storage opaque attribute.
I was going to experiment with your suggestion but 6 months later I cannot reproduce the original problem. I'm fairly sure I am using the same stream with its crazy bitrate. I'm sure we saw a significant use of malloc under set_opaque. I'm going to have to see if I can find anything in my logs.
Anyway your suggestion of min_size or maybe extra_size would probably be helpful.
It was for SRT, no?
Kieran
On Mon, Sep 22, 2025 at 3:02 PM JDarnley @.***> wrote:
JDarnley left a comment (Upipe/upipe#1067) https://github.com/Upipe/upipe/issues/1067#issuecomment-3319228801
I was going to experiment with your suggestion but 6 months later I cannot reproduce the original problem. I'm fairly sure I am using the same stream with its crazy bitrate. I'm sure we saw a significant use of malloc under set_opaque. I'm going to have to see if I can find anything in my logs.
Anyway your suggestion of min_size or maybe extra_size would probably be helpful.
— Reply to this email directly, view it on GitHub https://github.com/Upipe/upipe/issues/1067#issuecomment-3319228801, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABDEECK3CCUVCRSUW5PPSL3T76OFAVCNFSM6AAAAABZFHYCDOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGMJZGIZDQOBQGE . You are receiving this because you are subscribed to this thread.Message ID: @.***>
I'm fairly sure it was this stream. Every packet going through upipe_udp_source would get the attribute the pipe doesn't know if the packet is being used for RTP or SRT. This is why I had the impression it was proportional to packet rate (bitrate).