NYU Journalism NYC




SEARCHING THE WEB

BEFORE YOU BEGIN:

http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/ThingsToKnow.html: Before you begin searching, you need to know what you've been missing. The Web comprises some three billion documents (and counting); search engines only search a fraction of that. Meanwhile, the deeper Web---the so-called "invisible" Web---remains inaccessible to most engines.

HABITS OF HIGHLY EFFECTIVE WEBSURFERS:

http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/Strategies.html: This handy cheat sheet of search strategies is one of the many reasons the award-winning UC Berkeley Library tutorial on websearching is, in its own words, "the most heavily used and linked to tutorial on web searching that exists anywhere in the world." There's even a supercool form that helps you determine the best search strategy by answering a series of multiple-choice questions. Print it out at http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/form.pdf.

After you've read Berkeley's search strategies document, review its equally helpful "Search Tools" page, at http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/ToolsTables.html.

"Boolean" searching is an effective means of pinpointing hard-to-find data. Read Berkeley's page on Boolean tricks, at http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/Boolean.pdf.

Now you're ready to search.

START YOUR ENGINES:

www.google.com: There are gazillions of engines, but there's only one Google. For most journalists---hell, for most websearchers---Google rules.

Quick Google Tips:

1. Always use Google's "Advanced Search" page, at http://www.google.com/advanced_search?hl=en, which allows you to search by phrase, with specific terms included or excluded, for greater accuracy.

2. Using the drop-down menu under "results," always set any engine's results per page for the maximum allowed (in Google's case, 100). That way, you can scan a large number of results at a glance, picking out useful hits.

http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/Google.html: Get the most out of Google.

http://toolbar.google.com/: Highly recommended: Download Google's toolbar and add it to your Web browser. This will enable you to search within any site on the Web, whether it has its own search engine or not.

Of course, Google's not the only game in town.

http://www.AllTheWeb.com: According to Web expert Jonathan Dube, "AllTheWeb has indexed more Websites than Google. Does this make AllTheWeb.com better? Not necessarily. It uses different search technology -- putting more emphasis on keywords, while Google ranks sites based on how often other sites link to them. They'll each give you slightly different results. I still believe Google is the best search engine, but AllTheWeb is a good one to turn to when Google lets you down. AllTheWeb also has a couple of special searches Google doesn't: [For example,] You can search FTP files -- which basically means searching millions of files that are on computers connected to the Internet, but are not actually Web pages."

http://www.nelsonsearch.org/index.htm: Nelson, a "concept engine" developed by the journalism site Poynter.org, searches only those sites "that are most likely to be useful to journalists," which in Poynter's opinion consists of news sites (a selection of more than 200 print, broadcast, and online news sites from around the world), journalism organizations, subject guides (174 sites relevant to particular beats in the newsroom), and, of course, Poynter's own site. Worth a whirl.

BEYOND GENERAL SEARCHING:

http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/InvisibleWeb.html#How: Beyond the Web, there's the Deep Web, sometimes known as the "invisible" Web. Largely inaccessible to most search engines, it can be found through gateway pages, subject guides, discussion groups, and virtual reference libraries. This page offers a quick, schematic overview of these resources, many of them unknown to the average searcher.

LINKS LISTS:

http://www.geocities.com/mike_reilley_2000/newswriting/search.html: "Online Resources for Reporters and Editors," organized into sections on Search Engines, Subject Directories, Meta-Search Tools, Search Guidelines, Weblogs and the Invisible Web. Well-organized, cleanly laid-out, and carefully chosen. A wonderful resource.

SUBJECT GUIDES AND STARTING PAGES:

First, read Berkeley's highly informative page on subject directories, at http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/SubjDirectories.html.

Then check out:

http://reporter.umd.edu/: A journalist's guide to the Internet, by the associate dean of journalism at the University of Maryland at College Park and author of A Journalist's Guide to the Internet (Allyn & Bacon - 1999). Some juicy stuff here, arranged thematically: Courts and the Law, Federal Govt., Records and FOIA, and so on.

http://nilesonline.com/data/: NilesOnline, a site run by LATimes.com producer Robert Niles, is "an essential resource for journalists and other Web users...the best starting point for stats if you are on deadline." The site includes "annotated outside links" to "statistics on various topics, divided by categories such as crime, economics, education, health, law & politics and military." It also features "'Statistics Every Writer Should Know,' a basic math primer on topics such as 'per capita,' 'percent,' 'median,' etc." (Sreenath Sreenivasan)

http://www.statistics.com/: "Statistics.com provides links to data about all sorts of topics (business, military, education, sports, among others). The site, run by stats maven Peter Bruce, doesn't provide the data itself, but links out the best places on the Web for the respective stats." (Sreenath Sreenivasan)

http://www.journaliststoolbox.com/: "The Journalist's Toolbox features more than 13,000 web sites helpful to the media and anyone else doing research. Use the pulldown menu or search engine to locate information from a variety of beats and news industry-related topics."

http://www.ejc.nl/jr/search/index.html: The European Journalism Centre's page on websearch techniques also has some superb collections of links for "Getting Background," "Finding Source Documents," "Keeping Up on Your Beat," and the like.

FREQUENTLY USED SITES:

CAMPAIGN FINANCE/GOVERNMENT CONTRACTS:

http://www.campaignfinance.org/: Powerful tools for accessing information about campaign contributions, the awarding of government contracts, and more, from Investigative Reporters and Editors, Inc. NOTE: Some areas, such as the Federal Contracts Database http://www.campaignfinance.org/fpds/index.html, require paid membership. A complete listing of government database info for sale from IRE can be found at http://www.ire.org/datalibrary/databases/index.html.

CRIME:

http://www.ojp.usdoj.gov/bjs/: U.S. Dept. of Justice Statistics site. Raw data and the reference tools you'll need to analyze it, side by side. Access to state and federal statistics in 10 categories: Crime and victims, law enforcement, prosecution, the federal justice system, criminal offenders, courts and sentencing, special topics, corrections, expenditures and employment, and criminal record systems, which covers topics such as background firearm checks and sex offender registries. Main page includes a useful "Key Facts at a Glance" button, for quick access to statistical reports on subjects such as "Trends in Felony Convictions in State Courts."

CENSUS INFO:

http://www.census.gov/: Fast facts on U.S. income, poverty, genealogy, housing, economic data, foreign trade, and more.

EDUCATION:

Ask Eric

ENVIRONMENT:

http://www.epa.gov/enviro/html/ef_overview.html: Environmental Protection Agency's Envirofacts page: Find out about industrial facilities in your city that have been discharging toxic chemicals.

EXPERTS:

http://www1.profnet.com/profnet_home/index.html: "Good sources are hard to find. That's why there's ProfNet, a collaborative of information officers linked by Internet to give journalists free, convenient access to expert sources. We represent 8,000 organizations in North America and Europe. There's no limit to the subjects we cover."

http://www.ejc.nl/jr/search/findpeop.html: Great page of links to informed sources as well as technologies for tracking down contact info for government officials, authors, and other sources, including their e-mail addresses and phone numbers.

http://www.ibiblio.org/slanews/internet/experts.html: Encyclopedic collection of links to sources and experts, from universities and thinktanks to "experts clearinghouses" and an invaluable inventory of experts by field.

Experts Resource ListExhaustive list of U.S. and International sources list maintained by news researcher Kitty Bennett of The St. Petersburg Times for her newsroom's intranet.

"Web Tips" columnist Jonathan Dube has an excellent column on trolling newsgroups for man-on-the-street opinion and self-appointed experts at http://www.poynter.org/column.asp?id=32&aid=3131. See, also, "Finding Real People for Real Stories," at http://www.poynter.org/column.asp?id=32&aid=33707.

Multicultural.com/experts: "Features excerpts from The Source Book of Multicultural Experts, a guide to companies and experts in various markets, including Hispanic, Asian-American, African-American, gay and lesbian, and others. The book features listings sorted by each market as well as by industries such as advertising, healthcare, telecom, and retail. To get an idea of what's available in the book, see the alphabetical list of contact information for the various organizations here." (Sreenath Sreenivasan)

www.amazon.com: Often overlooked as a journalistic research tool, Amazon is a readymade mall of experts---a one-stop-shopping source of commentators on every subject imaginable. Let's say you're writing an article on biological weapons. Plug the word into Amazon's search engine and---voila!---dozens of hits, with the best-selling titles at the top. Clicking on one of those titles will give you the author's and publisher's name. Armed with the publisher's name, you can search for the publisher's website, which will usually give you the number of the publicity department. Call them, find out who handles media requests for the author you're interested in, and approach that publicist about an interview with the author. As always, use common sense: an unknown or minor-league writer with a new book to flog is going to be more likely to consent to an interview than a reclusive literary legend whose media sightings are rarer than sharp-focus photos of Yeti.

http://www.webring.org/rw: Webrings---a collection of sites, bound by a common theme, that can be browsed using the FORWARD and BACK buttons on your browser---are a great way to ramp up your knowledge of a subject or subculture and to troll for experts in that field. Often, the site's owner/editor makes an excellent source on the subject of her site. Failing that, she can usually point you toward one.

FOREIGN AFFAIRS (FAST FACTS ABOUT COUNTRIES)

http://www.worldinformation.com: "This London-based site is a division of World of Information, a business information company that's been around the old-fashioned way for 25 years. The site offers many useful features, including basic facts on each county, along with a visitor's guide, maps, current news links, and Web resources. I have used this to find basic information about countries I don't know enough about or to start research about a nation or region. As with any Web resource, it's good to double-check the information, especially since the amount and quality of data on all 200+ countries is not uniform. While the 'key facts' section is usually current (India's brand-new President is listed), the 'Web resources' and 'reading list' for many countries are not. Afghanistan, e.g., has its key facts updated, but some of the historical and other information is woefully inadequate. Still, this is a good resource worth checking out." (Sreenath Sreenivasan)

http://www.economist.com/countries: "The site of The Economist magazine (or 'newspaper,' as it likes to call itself) offers briefings on 60 countries. The backgrounders include profiles, financial statistics, news updates, and suggested Internet resources for each country -- all from the unique viewpoint of this respected British publication. Most of the information is available free of charge, with premium content marked with a E+." (Sreenath Sreenivasan)

http://www.cia.gov/cia/publications/factbook: "No matter what your opinion of the CIA, there's no doubt that journalists in the U.S. and abroad have been using this online resource for basic facts on countries. It hasn't been updated since Jan. 2001, but is still a handy resource on deadline." (Sreenath Sreenivasan)

GOVERNMENT: Bureau of Labor statistics, Citylink, FBI, Federal Election Commission, Fedworld, USDA, U.S. Department of Commerce, U.S. Supreme Court

HEALTH STATISTICS (U.S.):

http://www.cdc.gov/nchs/default.htm: The CDC's National Center for Health Statistics. Tabulated state-by-state health data; info on how to obtain birth, death, marriage, and divorce certificates; and much, much more. Vast.

National Institute of Health

LISTSERVS:

http://www.lsoft.com/lists/listref.html: CataList, the catalog of LISTSERV lists, enables you to browse any of the 53,559 public LISTSERV lists on the Internet, search for mailing lists of interest, and get information about LISTSERV host sites. Topica also offers a searchable database of listservs and discussion groups, although many of them are basically read-only newsletters, hosted by Topica.

MEDIA:

http://newslink.org/: U.S. News called this comprehensive list of links to sites for mainstream and alternative news publications and TV/radio outlets, both national and international, an "essential resource." Once you've found the outlet you're looking for, click through to the site. In most cases, it will be searchable; in many cases, full-text articles and transcripts will be available at no cost, or for a nominal fee.

http://www.findarticles.com/PI/index.jhtml: "The Web's first FREE article search." Lexis-Nexis for the masses.

http://www.ibiblio.org/slanews/internet/archives.html, hosted by the Special Libraries Assocation, also offers a list of links to U.S. newspaper sites, alphabetical by state, with information about whether they're free or what they cost.

For non-U.S. newspaper sites, see http://www.ibiblio.org/slanews/internet/ForArchives.html.

The Library of Congress also has a rich "newspaper archives/indexes/morgues" page, at http://lcweb.loc.gov/rr/news/oltitles.html.

http://www.bartleby.com/: "Libraries aren't the only places you can read books for free. If you want to read classics, this useful site offers full-text access to thousands of literary works, in addition to some contemporary reference volumes. And journalists can use these works to fact-check quotes, check attributions, etc.
Bartleby has what it calls "great books online." You will also be able to access such titles as The Columbia Encyclopedia (2001), The Encyclopedia of World History (2001), Roget's Thesaurus and Bartlett's Familiar Quotations." (From "Reading Books Online"
by Sreenath Sreenivasan.)

http://consumer.tveyes.com: In his October 8, 2002 "Web Tips" column, Sreenath Sreenivasan flags this "unusual, free site that...allows you to keep tabs on what's said on television. Enter a keyword and the service will instantly e-mail you a transcript as soon as it is spoken on a station it tracks. Enter...a keyword and when a correspondent or anchor mentions it, you will get an e-mail (about 170 words long) which tells you what was said about [your keyword]." The site lists the Top 10 keywords being tracked. Sree notes, "This is based on closed captioning technology, so is only as reliable as those captions' contents. And it certainly doesn't cover all the networks, so it's not exhaustive." There's a free consumer version as well as a professional version that provides access to archives, among other features.

http://TV4all.com/portal.htm: "Broadband streaming of 250+ television services around the world. VERY enlightening if you want to get an idea of how other countries view various stories." (Sree)

NEWSGROUPS/USENET:

http://groups.google.com/: Access to Usenet discussion forums and an archive of 20 years' worth of Usenet posts. According to Google, it has "fully integrated the past 20 years of Usenet archives into Google Groups, which now offers access to more than 700 million messages dating back to 1981. We believe this to be the most complete collection of Usenet articles ever assembled and a fascinating first-hand historical account."

Jonathan Dube, "Web Tips" columnist for Poynter.org, has a useful column on using newsgroups for research, at http://www.poynter.org/column.asp?id=32&aid=3128.

PHONE NUMBERS: The following links are cribbed from "Looking For Phone Numbers," by Columbia University Journalism Professor Sreenath Sreenivasan:

A word of caution from Sree: "Sometimes, Web listings may be out of date."

REMOTE SENSING:

http://www.american.edu/radiowave/: Arcane as hell, so much so that you'll probably never use it unless you end up prying into "black-budget" pentagon secrets or looking for the remains of the autopsied aliens in Area 51, but this journalists' resource is so brutally cool you just have to bookmark it. Links to sources of satellite imagery, U.S. government earth-sensing programs, digital elevation data, archeological imaging and ground-penetrating tools, and more.

NOW YOU'VE FOUND IT; CAN YOU BELIEVE IT? UNTANGLING FACT AND FICTION ON THE WEB.
The Web has revolutionized human knowledge by making oceans of information instantly navigable and, more radically, connecting a zillion previously unconnected dots of information into a massive constellation of knowledge. It's also a bog of misinformation, disinformation, falsehoods, and hoaxes.
Every reporter should use the excellent "Web Page Evaluation Checklist" at http://www.lib.berkeley.edu/TeachingLib/Guides/Internet/EvalForm.pdf to learn how to separate the dirty data from the hard facts.
http://lib.nmsu.edu/instruction/eval.html: Another superb, in-depth article about evaluating information you find on the Web.

HOAXES, RUMORS, AND DISINFORMATION:

http://www.truthorfiction.com/signs.htm: Do e-Rumors have recogizable characteristics that should enable the vigilant reporter to spot them? Every Web reporter should memorize this checklist.

http://hoaxbusters.ciac.org/: "The BIG list of Internet hoaxes."

http://urbanlegends.about.com/library/blhoax.htm: About.com page devoted to Internet hoaxes, e-mail rumors and urban legends. Entertaining and enlightening.

http://www.poynter.org/column.asp?id=32&aid=9344: If it's in The New York Times, it must be true, right? Maybe not. In "URL Cautionary Tales," cyberjournalist Jonathan Dube describes how Web hoaxers can fake official-looking URLs.

(Dube has also written a great column on how to truncate URLs to find documents hidden in non-password-protected areas of official websites. "Playing With URLS," at http://www.poynter.org/column.asp?id=32&aid=8849, is full of clever tips for doing end-runs around corporate and government firewalls. Says Dube, "By altering a few characters, removing the last part of the URL or simply guessing, you might just end up with a scoop.")

JUST FOR FUN (BUT ALSO USEFUL FOR TREND-SPOTTING):

http://www.metaspy.com/: LIVE search display. "Ever wonder what the rest of the world is searching for? Catch a glimpse of some of the searches being performed on MetaCrawler®, at this very moment!" A peek at the keywords people are plugging into one of the Web's most popular search engines, automatically refreshed every 15 seconds. Available in censored and uncensored versions.

http://50.lycos.com/: Every day, Lycos provides a window on what the world is searching for, based on keyword searches performed at Lycos. Especially useful for buzz-hungry journalists who need to know what's hot and what's on, in the minds of the webcrawling masses.

http://50.lycos.com/020400.html: The same info "raw," although how "raw" can it be if they've "taken out the naughty bits"?

http://www.searchenginewatch.com/facts/searches.html: Comprehensive list of ways to find out what other people are searching for, from "Ask Jeeves Peek Through a Keyhole" to Yahoo's "Buzz Index."

KEEPING YOUR CHOPS IN TOP FORM:

Subscribe

to the "Web Tips: From the Pros" (twice weekly) e-mail newsletter at http://www.poynter.org/profile/login.asp.

Browse

http://sree.net/stories/web.html: Columbia professor of online journalism and noted search-engine jock Sreenath Sreenivasan's justly famous "Smarter Surfing" Links: "Links for various categories of sites, annotated for your surfing pleasure." To receive a free advance copy of Sree's newsletter each month, send an e-mail to sreetips-subscribe@yahoogroups.com (just one message a month).

Browse

http://www.editorandpublisher.com/editorandpublisher/features_columns/columns.jsp: Archive of "Reporter's Digital How-To" columns by EditorAndPublisher.com's Charles Bowen. In each column, Bowen profiles an online resource useful to journalists, from the news-exclusive search engine RocketNews to the Internet Archive, a graveyard for dead websites.