DigitalNZ's story begins back in November 2006 when two initiatives were presented to Cabinet:
These two initiatives were then grouped together to become Digital New Zealand, which became part of the Government's Digital Content Strategy in September 2007. DigitalNZ was one of the two Digital Content Strategy initiatives to be led by the National Library of New Zealand, the other being the Aotearoa People's Network Kaharoa.
DigitalNZ was fully established and the first team members were assembled. Fiona Fieldsend as Programme Manager, Andy Neale as Technical Lead, and Lewis Brown as Policy Lead. Our numbers were few, but our aspirations were great. We have worked with some incredible people over the years... and you can read all about our team and alumni.
'Let’s make New Zealand’s digital content more accessible to our communities,’ we said. And, ‘Let’s promote open standards, formats, and interoperability so that we can maintain our heritage over time.’ There was obviously a lot to get on with.
At the beginning of August 2008 the DigitalNZ team held the kick-off for its first development sprint. The software development team was charged with building up the search infrastructure, websites and tools.
DigitalNZ was to become the National Library's first fully Agile Scrum project, and the team has been running fortnightly development sprints ever since. We recently determined that as of February 2014, we have completed 134 concurrent sprints (and still running...).
Our very first scrum master was Eduard Liebenberger, who did an amazing job wrangling our founding development partners at Boost New Media, 3Months, and Codec.
In November 2008, we launched Coming Home at a National Digital Forum event at the Auckland Museum, our first project as DigitalNZ.
Coming Home was a search engine of 30,000 aggregated digital items relating to the Armistice that ended the First World War, and was the first example of how we could pool together NZ material in a focused way. This is what the home page looked like:
We also launched the Memory Maker video remix tool in partnership with the Auckland War Memorial Museum. Memory Maker allowed people to drag-and-drop images, audio, and video onto a timeline to create their own multimedia stories. It was based on technology developed by Ideum.
On 3 December 2008 DigitalNZ officially launched www.digitalnz.org which, at the time, looked like this:
Key to the launch of DigitalNZ was the creation of the open Application Programming Interface (the API) which launched a data service that made the many different sources of NZ content available via a single search. This was one of the first examples in the world of an API being developed with the intention of aggregating a nation's digital content, and is believed to be the NZ Government's first RESTful and public API.
Alongside the launch of the API was the launch of Classic Search, a website demonstration of the API in use that pooled together material that the likes of Google and other internet search engines couldn't reach. We have fond memories of Classic Search because it showed everyone what was possible when we pooled our resources together.
We didn't stop there! In June 2009 we launched 'Make it Digital' our one-stop shop for questions, advice, and ideas for creating digital content. In the original vision for DigitalNZ, a $20 million contestable fund was to be made available to support increased digitisation activity in NZ. That never eventuated, so we instead thought about how to support community activity, and 'Make it Digital' was the team’s response to the very complicated digitisation guides and standards that previously existed.
The guides are highly regarded, with popular advice including a scorecard to assist for the digitisation selection process, an easy-to-follow copyright status flowchart, the NZ public domain guide, and advice on how to digitise family history and whakapapa.The advice is modelled on a Digital Content Lifecycle that we developed, and it explains how there are many aspects of making digital material available and useful.
The service also included a voting tool where the community could vote and comment on what NZ material should be digitised. It was a way to bring a customer voice to institutional selection processes, and the suggestions and comments provided a unique insight into people's views at the time. On the basis of public comment, the National Library of NZ in fact decided to raise the priority of digitising the Appendix to the Journals of the House of Representatives (AtoJs). DigitalNZ provided seed funding for this effort and the Library subsequently launched AtoJs in 2010.
DigitalNZ encouraged other organisations to use the open API to build search services using DigitalNZ’s metadata. When Housing Research NZ approached DigitalNZ in 2009 about using their API, we said, ‘Yes, of course!'.
So, Housing Research NZ built a search engine where visitors can find information relating specifically to housing in New Zealand. This was one of the first examples of an organisation using the powerful DigitalNZ API.
At the end of 2009, DigitalNZ was in full swing. We were bringing on board more and more content partners, the API was organising an ever-increasing set of metadata, and other organisations were building interesting applications with it. Things were really taking off! 1,385,000 items were available in DigitalNZ Search, which included 387,312 images, 4,145 videos, and 62,400 research papers.
For the first time, DigitalNZ started to think about how value could be added to the corpus of metadata. An initial prototype was developed for geo-tagging, this allowed longitude and latitude co-ordinates to be created by pinning items to a map. As our first foray into crowdsourcing, it was a useful experiment that taught us a lot and lay the groundwork for future efforts to improve metadata quality.
The service was really put to the test in October 2010 when DigitalNZ brought in the entire collection of Papers Past, the National Library’s collaborative digitised newspaper collection. Bringing the total number of searchable items to 19,202,582. This included 663,013 images, 5,661 videos, and 112,396 research papers. The digitised collections of historic NZ newspapers is one of the most popular sources of information for family historians and researchers in NZ, and the bringing in of Papers Past was a huge milestone. A little known fact is that the National Library uses the DigitalNZ API to provide a data feed of Papers Past for developers who may want to data mine this incredible record of NZ history.
2010 was a busy time. DigitalNZ was working with new content partners, spreading the word about DigitalNZ services, and facilitating the ‘Make it Digital’ awards which provided micro-funding to local digitisation projects. To top all this off, we decided to launch the Mix & Mash competition, to encourage the creative use of reusable NZ content and data. Pitched as the 'Great NZ Remix and Mashup Competition', it was NZs answer to the wave of such competitions around the world. There were some incredible 2010 winners, including a NZ walking track guide, visualisation of power usage, a mobile app to support yacht racing, an MP playing card game for iPad, a visualisation of NZ tax receipts, an incredible remix of Katherine Mansfield poetry with illustrations, posters, remixed music videos, and cartoons galore!
We are still incredibly grateful to the founding sponsorship for Mix & Mash from InternetNZ, Google, Microsoft, Boost, Codec, Department of Conservation, Pixton, MusicHype, the National Library of New Zealand, and Creative Commons Aotearoa New Zealand.
2011 turned out to be a year of consolidation and maintenance. Our hands were full with updating the 'Make it Digital' guides, microfunding of digitisation, Mix and Mash, and the scaling of our infrastructure to support our ever growing content partners. We did however migrate the NZ Research service to the DigitalNZ platform to maintain access to aggregated research outputs. In August the second Mix & Mash competition was launched at the 2011 Orcon Great Blend event in Auckland, and in September both the homepage and search experience of www.digitalnz.org was redesigned and updated.
Following the devastating earthquakes that struck Christchurch in September 2010 and February 2011, the University of Canterbury were looking to establish a federated digital archive of material relating to the disasters, to 'preserve the memories and experiences of the people of the Canterbury region'. We were more than happy to help, and were delighted when UC CEISMIC was built using the DigitalNZ API, and launched in May 2012.
Early in 2012, the team were getting concerned about the amount of material being collected: ‘DigitalNZ is getting bigger and bigger. Potentially, it could grow forever. It needs a more human connection. How can we help people connect to this overwhelming sea of digital material?’
‘Sets’ were the idea that we came up with as a way for people to curate, collect, and share their favourite treasures. Now, DigitalNZ users can log in to www.digitalnz.org, collect together interesting items, and keep them all in one handy place. Here’s one of the first sets ever made, about 1980s New Zealand:
In November 2012 the National Library reopened its freshly redesigned building. The Lifelines touch-table was one of its new, exciting offerings. After the opening, visitors eagerly pored over digital material relating to different regions of New Zealand, not at all realising that its service is powered by DigitalNZ's API.
For that matter, most people would never know that the National Library of New Zealand relaunched other key services on top of the DigitalNZ services as well. Their digital front door at natlib.govt.nz, Search Stations for onsite researchers, and their AV Pods all run on top of DigitalNZ.
In 2013 we rebuilt our core metadata harvesting tools, taking everything we had learned over five years, and building our second generation of technology. Our harvesters are the tools we use to collect data from content partners. They now work much faster, in real-time, and at significant scale. We call our harvesting engine 'Supplejack'.
In April 2013, our Content Analyst, Dan Charles, noted that an update of the V. C. Browne aerial photographic collection tipped the number of images in DigitalNZ to over 1 million.
June 2013 saw another important initiative: thanks to five content partners—Victoria University of Wellington Library, New Zealand Electronic Text Collection, The University of Auckland Library, Palmerston North City Library, Kete Horowhenua, and Manatu Taonga: Ministry for Culture and Heritage—we were able to make 200,000 metadata records available for commercial re-use. This was the first of what we hope will be a number of commercial releases of our metadata holdings for people to use and build with.
In 2014 we celebrated our sixth birthday by announcing an exciting new 150th content partner, Radio New Zealand, who joined DigitalNZ with the pleasingly similar number of 150,000 audio recordings and interviews from their wide range of programmes.
We also made the activities of DigitalNZ more transparent by publishing our year plans, our quarterly work plans, and the minutes from the DigitalNZ Advisory Board meetings on our website. Now those who are interested can track our progress and find out about where we are focussing our efforts during the year.
In October we reviewed and then deprecated some of our older services. We bid a fond farewell to Matapihi, an aggregated search service which was a predecessor to DigitalNZ, as well as two older versions of the DigitalNZ API. You can see a full list of our retired services here.
In November we worked with the Digital Public Library of America to stage the inaugural GIF IT UP challenge. This was a fun challenge calling for GIFs made from openly licensed and public domain material found on either DigitalNZ or the DPLA. The challenge received over a hundred entries from around the world, including this one from Nono Burling which reused material from the University of Southern California Libraries.
2015 saw us achieve a significant infrastructure milestone when we migrated our harvesting processes and search APIs to run solely off the new open source Supplejack software. Also, Nga Taonga became the first organisation to use Supplejack for their own purposes, for the new aggregated search on their website.
Early in 2015, members of DigitalNZ and a reference group from around the country met to discuss the future of the Kete digital repositories. The existing Kete software is near end of life. It was agreed that DigitalNZ should test bringing National Library of New Zealand's existing APNK Kete features into an updated DigitalNZ service. At the National Digital Forum conference at the end of the year, we demonstrated a prototype of this service, alongside new plans for sets.
Late in 2015 we launched the Concepts API, the first step in our People and Places data enrichment, which will allow for linkages between people and items. The first iteration of the Concepts API was launched with the people authority records from five different collections. You can read all about it in this blog post.
At the end of the year we hosted a number of our content partners and other interested people at a DigitalNZ Day at the National Library. During the day we shared the results of the values research we had recently completed and we also took the chance to workshop ideas with the group about the future of DigitalNZ and areas where we should focus our efforts.
As history continues to be made... we'll post it here!