Generating three-dimensional human models representing two-dimensional humans in two-dimensional images

Fuente: Wipo "digitalization"
The present disclosure relates to modification of two-dimensional (2D) images via scene-based editing using generated three-dimensional (3D) representations of the 2D images. The disclosed systems may generate 3D human models from 2D images to modify humans appearing in the 2D images. This is achieved by using neural networks to extract 2D pose data 6006 from a 2D image of a human and further extracting 3D pose data 6010 and 3D shape data 6012 from the 2D image. A 3D model 6014 of the human extracted from the 2D image is generated using the 2D pose data, the 3D pose data and the 3D shape data. Using hand-specific bounding boxes 3D hand pose data may be identified and modified, thus enabling the 3D human model to be modified accordingly to generate an updated 2D image corresponding to a modified pose (see figure 62). Also disclosed are systems for generating 3D representations from 2D images to generate and modify shadows in the 2D images according to various shadow maps (see e.g. figure 40). 3D representations of 2D images may also be used to provide scene scale estimation via scale fields of the 2D images (see e.g. figure 50). In some embodiments, the disclosed systems utilise 3D representations of 2D images to generate and visualise 3D planar surfaces for modifying objects in 2D images (see e.g. figure 67). The disclosed systems further use 3D representations of 2D images to customise focal points for the 2D images (see e.g. figure 74C).