Feat: Add support for fastapi lifespan events

[robTheBuildr]

🚀 Feature

Add the ability for developers to customize the FastAPI lifespan, startup, and 'shutdown` event handlers.

Motivation

Users can use this to setup logging or resource management.

Pitch

There are 2 options here.

1. Add a function attach_event_handler(event: str, func: Callable) which takes either 'lifespan', 'startup' or 'shutdown' as the event and applies the given logic.

2. More explicit and abstracting away FastAPI Lifespan event knowledge from the user, add during_lifespan(func: Callable), on_startup(func: Callable), and on_shutdown(func: Callable) that the user can call on Server to apply these lifespan handlers.

Alternatives

Add custom template startup and shutdown event handler for logging and resource management. This will get cumbersome.

Additional context

[github-actions[bot]]

Hi! thanks for your contribution!, great first issue!

[aniketmaurya]

hi @robTheBuildr, we are using the FastAPI lifespan to manage our parallel inference workers. May I know more about your use case and what you are trying to achieve with lifespan?

[williamFalcon]

@robTheBuildr what will be helpful is a reproducible code snippet we can run that shows something you are trying to do, but not able to do with litserve

[vrdn-23]

@williamFalcon @aniketmaurya Hi! I'm new to the project and this seems interesting to our team for the use-cases we are working on.

what will be helpful is a reproducible code snippet we can run that shows something you are trying to do, but not able to do with litserve

In our case, we would essentially like to be able to add custom middleware to the FastAPI object so that we can track internal metrics and other things that is necessary for our infra.

An example of how that would look like is this:

_app = app or FastAPI()
_app.add_middleware(TrackingIdMiddleware)
_app.add_middleware(InstrumentedMiddleware)
_app.add_middleware(AccessLoggingMiddleware)
_app.add_middleware(ProxyHeadersMiddleware)

I'm not sure if it's possible to also allow users to pass in their own FastAPI app that has it's own lifespan and then append a lifespan that the LitServer uses to it.

[aniketmaurya]

hi @vrdn-23, you can track the progress for adding middleware in #228

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[aniketmaurya]

closing this since we are already using lifespan