GraphiteEditor
AI models for graphics editing
Below is a list of AI model disciplines that will be useful tools in the graphics editing process. Feel free to comment with ideas for more items missing from this list.
- Image generation (to turn inputs like text, images, masks, or other control data into a desired image)
- Infill (to generate the content in missing areas based on a mask, or uncrop an image)
- Style transfer (to adapt the subject of one image to the art style of another image)
- Upscaling
- Segmentation (to automatically mask a subject or break a scene apart into multiple subjects)
- Depth estimation (to generate a depth map that can be used for many procedural effects)
- Decomposing into render channels like albedo, normal, depth, irradiance, roughness, metalness (RGB↔X)
- Relighting (to change the direction and color cast of the lighting on a scene or subject)
- Novel view synthesis (to alter the perspective angle or FoV of a subject)
- Altering a scene's focus or deconvolving blur
- Un-smearing a motion-blurred image
- Noise removal (sensor noise, JPEG artifacts)
- SDR to HDR conversion by inferring the extra data that was outside the camera's dynamic range
- Recovering clipped pixels in overexposed scenes
- Color gamut extension by inferring true WCG colors of a scene beyond the range of the imaging device sensor
Can I possibly submit a GSOC Proposal for this exact mega-task? It seems like an interesting domain to which I'd like to contribute, can see starting points for 4 out of 5 pointers (except Novel View Synthesis), and would very much like to learn in these fields, and seems like the timeline matches with that of GSOC?
Though I can understand if the maintainers would like someone more specialized to be up for this task, if I get the opportunity, I'll definitely make it up to the quality standards maintained in this great project!
Let me know if it's possible and I'll start working on a proposal right away!
@CheeksTheGeek please join the Discord and discuss this with me there. The task is described somewhat further in https://graphite.rs/volunteer/guide/projects/student-projects/#machine-learning-architecture. Just to set expectations, I will note that this requires a decently advanced level of understanding in deep learning concepts, including a fluent understanding of how to use libraries like PyTorch or similar to implement concepts based on your readings of research papers.