DMOZ or DMoz (stylized dmoz in its logo; from directory.mozilla.org, an earlier domain name) was a multilingual open-content directory of World Wide Web links. The site and community who maintained it were also known as the Open Directory Project (ODP). It was owned by AOL (now a part of Yahoo! Inc) but constructed and maintained by a community of volunteer editors.
DMOZ used a hierarchical ontology scheme for organizing site listings. Listings on a similar topic were grouped into categories which then included smaller categories.
DMOZ closed on March 17, 2017, because AOL no longer wished to support the project. The website became a single landing page on that day, with links to a static archive of DMOZ, and to the DMOZ discussion forum, where plans to rebrand and relaunch the directory were being discussed.
, a non-editable mirror remained available at dmoztools.net, and it was announced that while the DMOZ URL would not return, a successor version of the directory named Curlie would be provided. DMOZ, ODP, and Curlie were considered synonymous by 2018. Curlie was well established by 2022, using the hierarchy from DMOZ.
History
DMOZ was founded in the United States as GnuHoo by Rich Skrenta and Bob Truel in 1998 while they were both working as engineers for Sun Microsystems. Chris Tolles, who worked at Sun Microsystems as the head of marketing for network security products, also signed on in 1998 as a co-founder of Gnuhoo along with co-founders Bryn Dole and Jeremy Wenokur. Skrenta had developed TASS, an ancestor of tin, the popular threaded Usenet newsreader for Unix systems. The original category structure of the Gnuhoo directory was based loosely on the structure of Usenet newsgroups then in existence.
The Gnuhoo directory went live on June 5, 1998. After Richard Stallman and the Free Software Foundation objected to the use of Gnu in the name, GnuHoo was changed to NewHoo. Yahoo! then objected to the use of Hoo in the name, prompting a proposed name change to ZURL."Zurl Directory" Prior to switching to ZURL, NewHoo was acquired by Netscape Communications Corporation in October 1998 and became the Open Directory Project. Netscape released Open Directory data under the Open Directory License. Netscape was acquired by AOL shortly thereafter and DMOZ was one of the assets included in the acquisition.
By the time Netscape assumed stewardship, the Open Directory Project had about 100,000 URLs indexed with contributions from about 4500 editors. On October 5, 1999, the number of URLs indexed by DMOZ reached one million. According to an unofficial estimate, the URLs in DMOZ numbered 1.6 million in April 2000, surpassing those in the Yahoo! Directory.ODP and Yahoo Size Charts by ODP editor geniac DMOZ achieved the milestones of indexing two million URLs on August 14, 2000, three million listings on November 18, 2001, and four million on December 3, 2003. As of April 2013 there were 5,169,995 sites listed in over 1,017,500 categories. On October 31, 2015, there were 3,996,412 sites listed in 1,026,706 categories.
In January 2006, DMOZ began publishing online reports to inform the public about the development of the project. The first report covered the year 2005. Monthly reports were issued subsequently until September 2006.ODP reports by ODP volunteer administrator chris2001 These reports gave greater insight into the functioning of the directory than the simplified statistics provided on the front page of the directory. The number of listings and categories cited on the front page included "Test" and "Bookmarks" categories but these were not included in the RDF dump offered to users. There were about 7330 active editors during August 2006. 75,151 editors had contributed to the directory as of March 31, 2007.ODP Front Page . Retrieved August 15, 2006 As of April 2013, the number of contributing editors had increased to 97,584.
System failure and editing outage, October to December 2006 On October 20, 2006, DMOZ's main server suffered a catastrophic failure that prevented editors from working on the directory until December 18, 2006. During that period, an older build of the directory was visible to the public. On January 13, 2007, the Site Suggestion and Update Listings forms were again made available.dmoz.org technical problems resource-zone.com Retrieved July 17, 2017. On January 26, 2007, weekly publication of RDF dumps resumed. To avoid future outages, the system resided on a redundant configuration of two Intel-based servers from then on.The Hamsters' New Home , in: Open Directory newsletter issue Winter 2006. Retrieved December 26, 2006.
The site's interface was given an upgrade in 2016, branded "DMOZ 3.0", but AOL took it offline the following year.
Competing and spinoff projects As DMOZ became more widely known, two other major web directories edited by volunteers and sponsored by Go.com and Zeal emerged, both now defunct. These directories did not license their content for open content distribution."Terms of Use" "GO Network Terms of Service and Conditions of Use"
The concept of using a large-scale community of editors to compile online content has been successfully applied to other types of projects. DMOZ's editing model directly inspired at least three other open content volunteer projects: music site MusicMoz, an open content restaurant directory known as ChefMozChefMoz Fine Dining Menu , in: Open Directory newsletter issue Autumn 2003 and an encyclopedia known as Open Site.help on open-site.org Finally, according to Larry Sanger, DMOZ was part of the inspiration for the Nupedia project, out of which Wikipedia grew.
Logo history
Content Gnuhoo borrowed the basic outline for its initial ontology from Usenet. In 1998, Rich Skrenta said, "I took a long list of groups and hand-edited them into a hierarchy." For example, the topic covered by the comp.ai.alife newsgroup was represented by the category Computers/AI/Artificial_Life. The original divisions were for Adult, Arts, Business, Computers, Games, Health, Home, News, Recreation, Reference, Regional, Science, Shopping, Society, Sports and "World". While these sixteen top-level categories have remained intact, the ontology of second- and lower-level categories has undergone a gradual evolution; significant changes are initiated by discussion among editors and then implemented when consensus had been reached.
In July 1998, the directory became multilingual with the addition of the World top-level category. The remainder of the directory lists only English language sites. By May 2005, seventy-five languages were represented. The growth rate of the non-English components of the directory had been greater than the English component since 2002. While the English component of the directory held almost 75% of the sites in 2003, the World level grew to over 1.5 million sites as of May 2005, forming roughly one-third of the directory. The ontology in non-English categories generally mirrors that of the English directory, although exceptions which reflect language differences are quite common.
Several of the top-level categories have unique characteristics. The Adult category is not present on the directory homepage but it is fully available in the RDF dump that DMOZ provides. While the bulk of the directory is categorized primarily by topic, the Regional category is categorized primarily by region. This has led many to view DMOZ as two parallel directories: Regional and Topical.
On November 14, 2000, a special directory within DMOZ was created for people under 18 years of age.Kids and Teens Launches! Open Directory Project Newsletter, November/December 2000 Key factors distinguishing this "Kids and Teens" area from the main directory are: stricter guidelines which limit the listing of sites to those which are targeted or "appropriate" for people under 18 years of age; category names as well as site descriptions use vocabulary which is "age appropriate"; age tags on each listing distinguish content appropriate for kids (age 12 and under), teens (13 to 15 years old) and mature teens (16 to 18 years old); Kids and Teens content is available as a separate RDF dump; editing permissions are such that the community is parallel to that of DMOZ.
By May 2005, this portion of DMOZ included over 32,000 site listings.
From early 2004, the whole site was in UTF-8 encoding. Prior to this, the encoding had been ISO 8859-1 for English language categories and a language-dependent character set for other languages. The RDF dumps were encoded in UTF-8 from early 2000.
Maintenance Directory listings were maintained by editors. While some editors focused on the addition of new listings, others focused on maintaining the existing listings, and some did both. This included tasks such as the editing of individual listings to correct spelling and/or grammatical errors, as well as monitoring the status of linked sites. Still others went through site submissions to remove spam and duplicate submissions.
Robozilla was a Web crawler written to check the status of all sites listed in DMOZ. Periodically, Robozilla would flag sites which appeared to have moved or disappeared and editors follow up to check the sites and take action. This process was critical for the directory in striving to achieve one of its founding goals: to reduce the link rot in web directories. Shortly after each run, the sites marked with errors were automatically moved to the unreviewed queue where editors may investigate them when time permits.
Due to the popularity of DMOZ and its resulting impact on search engine rankings (See PageRank), domains with lapsed registration that were listed on DMOZ attracted domain hijacking, an issue that was addressed by regularly removing expired domains from the directory.
While corporate funding and staff for DMOZ diminished over time, volunteers created editing tools such as linkcheckers to supplement Robozilla, category crawlers, spellcheckers, search tools that directly sift a recent RDF dump, bookmarklets to help automate some editing functions, mozilla based add-ons,"ODPExtension" Mozilla based add-on, ODP Magic. formerly known as ODP Extension and tools that helped work through unreviewed queues.
License and requirements DMOZ data was previously made available under the terms of the Open Directory License, which required a specific DMOZ attribution table on every Web page that uses the data.
The Open Directory License also included a requirement that users of the data continually check DMOZ site for updates and discontinue use and distribution of the data or works derived from the data once an update occurs. This restriction prompted the Free Software Foundation to refer to the Open Directory License as a non-free documentation license, citing the right to redistribute a given version not being permanent and the requirement to check for changes to the license.
In 2011, DMOZ silently changed its license to a Creative Commons Attribution license, which is a free license (and GPL compatible).GNU Project: on the Creative Commons Attribution license
RDF dumps DMOZ data is made available through an RDF-like dump that is published on a download server, older versions are also archived there. New versions are usually generated weekly. A DMOZ editor has catalogued a number of bugs that are encountered in the DMOZ RDF dump, most importantly that the file format is not RDF. So while today the so-called RDF dump is valid XML, it is not valid RDF and as such, software to process the DMOZ RDF dump needs to be specifically written for DMOZ data.
Content users DMOZ data powers the core directory services for many of the Web's largest search engines and portals, including Netscape Search, AOL Search, and Alexa. Google Directory used DMOZ information, until being shuttered in July 2011.