Survey and challenges of story generation models - A multimodal perspective with five steps: Data embedding, topic modeling, storyline generation, draft story generation, and story evaluation

The story is the description of events in chronological order that have occurred between people. By delivering facts to the people reading the story, it enables them to feel emotions. Such a story is composed using the following method: each event is analyzed and a storyline is composed, which becomes a skeleton text by linking relationships between major events. As the content of users becomes more diverse, multimodal story composition has become more essential than unimodal text-based story composition. This paper discusses modality integration based on multimodal data types and type conversion for multimodal story composition. We propose a story-graph model to create a story based on the integrated analysis of various modal data. In terms of architecture, the proposed multimodal storytelling model consists of modal data and a topic modeling module that performs clustering based on cross-modal similarities and extracts a topic of clustered modalities. From the perspective of utilization, to visualize a story-graph, the proposed model summarizes nodes with a representative image. Furthermore, the latest techniques are discussed with respect to five main modules and twelve sub modules for story composition. Lastly, problems that can become issues when composing multimodal story modules are explained.

Automated summary

The story is a chronological description of events between people, delivering facts to evoke emotions in the readers. Multimodal story composition is becoming more essential with diverse user content, and this paper discusses modality integration based on various data types. A proposed story-graph model integrates various modal data for storytelling, clustering topics based on cross-modal similarities and summarizing nodes with representative images for visualization. The latest techniques for story composition are also discussed, along with potential issues in composing multimodal story modules.