IBM Develops Full-text Digitization System for National Diet Library of Japan

New collaborative technology improves the digitization of library collections worldwide

Select a topic or year

Tokyo, Japan - 01 Aug 2011: IBM (NYSE: IBM) today announced that it is helping the National Diet Library of Japan -- the country's only national library -- digitize its literary artifacts on a massive scale to make them widely available and searchable online by all information seekers.

The prototype technology, created by IBM Research, allows full-text digitization of Japanese literature to be quickly realized through expansive recognition of Japanese characters and enabling users to collaboratively review and correct language characters, script and structure. Additionally, the full-text digitization system is designed to promote future international collaborations and standardization of libraries around the world.

“Nearly two decades ago in his book Digital Library, Dr. Makoto Nagao, the director of the National Diet Library, shared his vision that digitized and structured electronic books will dramatically change the role of libraries and the way knowledge will be shared and reused in our society,” said Dr. Hironobu Takagi, who led the development of the prototype technology at IBM Research – Tokyo. “Until now, the breadth of the characters and expressions within the Japanese language had posed a series of challenges to massive digitization. In order to enable this transfer of knowledge from print to online, we realized the need for both machine and human intelligence to understand information in every form.”

Compared to other languages, which rely on just a few dozen alphabetical characters, Japanese is extremely diverse in terms of script. In addition to syllabary characters, hiragana and katakana, Japanese includes about 10,000 kanji characters (including old characters, variants and 2,136 commonly used characters), in addition to ruby (a small Japanese syllabary character reading aid printed right next to a kanji) and mixed vertical and horizontal texts.

Aside from ensuring quality recognition of Japanese characters, IBM researchers aimed to optimize the amount of time needed to review and verify the accuracy of the digitized texts. By introducing unique collaborative tools via crowdsourcing, the technology allows many users to quickly pour through the texts and make corrections at a much higher rate of productivity and efficiency.

“Through collaboration technology and user tools, we now have the potential to populate a global collection of literature and information,” said Dr. Takagi. “From small community libraries to national institutions, people everywhere can leverage this standardized system to help preserve and share their cultural works for years to come.”

The architecture of the full-text digitization prototype system provides the following two key collaborative features:

The full-text digitization prototype system was realized based on two streams of technologies. IBM researchers in Tokyo applied an innovative approach called Social Accessibility, which allows large groups of reviewers to work collaboratively via Web browsers regardless of location. Also, the COoperative eNgine for Correction of ExtRacted Text(CONCERT) technology -- developed by IBM Researchers in Haifa, Israel -- was leveraged to significantly improve productivity through the repetition of simple operations. 

Related XML feeds
Topics XML feeds
Financial news, company earnings, philanthropy, community service, human resources, sponsorship
Global Technology Services
Technology services that can deliver measurable business results.
Information Management, Lotus, Tivoli, Rational, WebSphere, Open standards, open source