SURFACE NORMAL ORIENTED TEXTUAL IMAGE GENERATION VIA CONTROLNET-AUGMENTED SPECIALIZED TEXT-TO-IMAGE GENERATION MODEL

Fuente: WIPO "tomato"
A method and system for surface normal oriented textual image generation via ControlNet-augmented specialized text-to-image generation model is disclosed. The textual image generation employs surface normals to guide the orientation of text, ensuring that each character's bounding box aligns with the underlying geometry of the surface. This is achieved by generating a character aligned character mask based on surface normal. Existing models struggle with text alignment on surfaces with angled perspectives. The method not only automates manual text placement and alignment but also enhances the visual coherence and realism of generated images. The approach substantially improves the state-of-the-art methods in text rendering, harmonization, and perspective blending.