Home

Welcome to the nutch-python wiki!

Getting started with Nutch-Python

Right now the API is evolving rapidly, but here is some code that should get you running.

Start the Nutch Server

Download and build the latest version of Nutch trunk (in a separate terminal).

1. git clone https://github.com/apache/nutch.git
2. cd nutch
3. ant runtime
4. cd runtime/local
5. ./bin/nutch startserver

Get your Nutch-Python script going

from nutch.nutch import Nutch
from nutch.nutch import SeedClient
from nutch.nutch import Server
from nutch.nutch import JobClient
import nutch

sv=Server('http://localhost:8081')
sc=SeedClient(sv)
seed_urls=('http://espn.go.com','http://www.espn.com')
sd= sc.create('espn-seed',seed_urls) 

nt = Nutch('default')
jc = JobClient(sv, 'test', 'default')
cc = nt.Crawl(sd, sc, jc)
while True:
    job = cc.progress() # gets the current job if no progress, else iterates and makes progress
    if job == None:
        break