Researchers of Digital Humanities (DH) at the Indian Institute of Technology Jodhpur (IITJ) are developing software to convert digital comic books to videos — Comic-to-Video Network (C2VNet). The C2VNet evolves panel-by-panel in a comic strip and eventually produces a full-length video (with audio) of a digitised storybook.
Along with the software, IIT Jodhpur researchers have also proposed a dataset titled “IMCDB: Indian Mythological Comic Dataset of Digitised Indian Comic Storybook” in English . This has complete annotations for panels, binary masks of the text balloon, text files for each speech balloon, narration box within a panel, and plans to make the dataset publicly available.
Chiranjoy Chattopadhyay, Assistant Professor, Department of Computer Science and Engineering, IITJ, said that the panel extraction model C2VNet has two internal networks to support the video creation.
“CPENet developed by the team gives over 97 per cent accuracy, and the speech balloon segmentation model SBSNet gives 98 per cent accuracy with fewer parameters. Both have outperformed state-of-art models. C2VNet is the first step towards the big future of automatic multimedia creation of comic books and bringing new comic reading experiences,” he said.