Kafka event is missing for staging complete

[ageorget]

Hi,

Currently when dCache asks to stage a file from tape, only one Kafka restore event is generated which corresponds to the submission request. This event seems to correspond to the pool log Pool pool-xxx asked to fetch file 000092B230A7D3314E969F6B5C41FDC86602

Would it be possible to add another Kafka event when the file is staged please? Like the pool log Staged 000092B230A7D3314E969F6B5C41FDC86602 from nearline storage. This is useful to get information about staging duration for admins.

One last thing, what does transferTime means in the restore event if this event is generated when the staging request is submitted?

Cheers, Adrien

[kofemann]

Hi Adrien,

The event is generated when the re request is submitted, but sent only after it's compete. Thus transferTime time is exactly the time difference between this two events.

tigran.

[ageorget]

Hi @kofemann Hmm I see, thanks for your answer. Well I think one information is still missing, the date when the staging is complete. I want to make visualizations of the staging activity like the number of staging events done in a time range but the date field indicates when the request was submitted to the pool. If you check my previous example, it can take time between the request is submitted to the pool and then processed (depending of the number of bringonline requests sent and the hsm -c:gets). Then there's another delay between the request is processed and the file is staged on disk.

As you can see, these visualizations are not really representatives of the staging activity, we can just see a peak of requests when the VO submitted their staging requests :

If it's not a hard work to add it, It would be really appreciated. Cheers, Adrien