VIDEO DOCUMENTATION [ 2 minutes ]

Introduce Cangjie's Poetry 仓 颉 的 诗

Cangjie’s Poetry is an intelligent multimodal system designed as a conceptual response to the future semantic human-machine reality. Inspired by an ancient Chinese legendary historian, Cangjie (c.2650 BCE), who invented Chinese characters based on the characteristics of everything on the earth, we trained a neural network that we call Cangjie, to learn the constructions and principles of over 9000 Chinese characters. After successfully training, Cangjie can interpret images through the lens of Chinese characters and produce new symbols constructed by Chinese strokes. Meanwhile, we implement a pre-trained model (DenseCap) to simultaneously generate localized descriptive sentences of images in natural language to create meanings for this symbolic system.

In the art installation, the Cangjie system captures surroundings using a camera and transforms the real world streaming into a cluster of ever-changing new symbols in real-time. Those novel symbols made of Chinese strokes tangled with the imagery captured by the camera in the installation are visualized algorithmically as an abstract pixelated landscape. The landscape dynamically moves, evolves, and writes poetry based on the image data of the live streaming captured by the camera. We project the visualization of the semantic landscape on the wall in the exhibition space as the first projection. In the meantime, Cangjie also generates descriptive sentences of surroundings based on its interpretation. Those sentences are designed as flowing poetry written in ink, assembled with real-time captured imagery fragments as the second projection in the space.

Cangjie is not only a conceptual response to the tension and fragility in the coexistence of humans and machines but also an artistic expression of a future language that reflects on ancient truths, a way to evoke enchantment in this artificial intelligence era. The interactivity of this intelligent system prioritizes ambiguity and tension that exists between the actual and the virtual, machinic vision and human perception, and past and present.

From learning over 9000 Chinese characters, Cangjie creates its own symbolic system. It observes the real-world, writes poetry using its symbolic system, and explains it in natural language to audiences. Just like the legendary historian Cangjie did nearly 5000 years ago. It continuously writes poetry with humans collaboratively, never ends, as long as the real world exists.

Visualization I Cangjie writes poetry using pixels of live streaming textures.

Visualization II Cangjie describes surroundings as a visual form of poetry.