python - Counting words from tokenized url -


very new python , hoping guys give me help.

i have book great war, , want count times country appears in book. far have this:

>>> __future__ import division  >>> import nltk, re, pprint >>> urllib import urlopen >>> url = "http://www.gutenberg.org/files/29270/29270.txt" >>> raw = urlopen(url).read()  >>> type(raw) <type 'str'> >>> len(raw) 1067008 >>> raw[:75] 'the project gutenberg ebook of story of great war, volume ii (of\r\nv' >>>

tokenization. break string words , punctuation.

>>> tokens = nltk.word_tokenize(raw) >>> type(tokens) <type 'list'> >>> len(tokens) 189743 >>> tokens[:10] //vind de eerste 10 tokens ['the', 'project', 'gutenberg', 'ebook', 'of', 'the', 'story', 'of', 'the', 'great'] >>>

correcting beginning , ending of book

    >>> raw.find("part i")     >>> 2629     >>> raw.rfind("end of project gutenberg")     >>> 1047663     >>> raw = raw[2629:1047663]     >>> raw.find("part i")     >>> 0

i unfortunately have no idea how implement book wordcount. ideal outcome this:

germany 2000 united kingdom 1500 usa 1000 holland 50 belgium 150

etc.

please help!

python has builtin method count substring in string.

from urllib import urlopen  url = "http://www.gutenberg.org/files/29270/29270.txt" raw = urlopen(url).read() raw = raw[raw.find("part i"):raw.rfind("end of project gutenberg")]  countries = ['germany', 'united kingdom', 'usa', 'holland', 'belgium'] c in countries:     print c, raw.count(c)

produces

germany 117 united kingdom 0 usa 0 holland 10 belgium 63

edit: eumiro right, doesn't work if want count exact word. use if want search exact word:

import re urllib import urlopen  url = "http://www.gutenberg.org/files/29270/29270.txt" raw = urlopen(url).read() raw = raw[raw.find("part i"):raw.rfind("end of project gutenberg")]  key, value in {c:len(re.findall(c + '[^a-za-z]', raw)) c in countries}.items():     print key, value

edit: if want linenumbers:

from urllib import urlopen import re collections import defaultdict  url = "http://www.gutenberg.org/files/29270/29270.txt" raw = urlopen(url).readlines()  count = defaultdict(list) countries = ['germany', 'united kingdom', 'usa', 'holland', 'belgium'] c in countries:     nr, line in enumerate(raw):         if re.search(c + r'[^a-za-z]', line):             count[c].append(nr + 1) #nr + 1 first line 1 instead of 0     print c, len(count[c]), 'lines:', count[c]

Comments

Post a Comment

Popular posts from this blog

java - Play! framework 2.0: How to display multiple image? -

i need display gallery of photos. here template: @(photos: list[photo]) @title = { <bold>gallery</bold> } @main(title,"photo"){ <ul class="thumbnails"> @for(photo <- photos) { <li class="span3"> <a href="#" class="thumbnail"> <img src="@photo.path" alt=""> </a> </li> } </ul> } and here controller method: public static result getphotos() { return ok(views.html.photo.gallery.render(photo.get())); } and here photo bean : @entity public class photo extends model { @id public long id; @required public string label; public string path; public photo(string path, string label) { this.path = path; this.label = label; } private static finder<long, photo> find = new finder<long, photo>( long.class, photo.class); public static list<phot...
Read more

gmail - Is there any documentation for read-only access to the Google Contacts API? -

i want read-only access user's google contacts. there stackoverflow member wanted this, , received following answer : yes, there is, use https://www.googleapis.com/auth/contacts.readonly scope , "view contacts". this exchange reference read-only access google contacts can find using google. simply using access token obtained attempt read usual google contacts api (i.e. https://www.google.com/m8/feeds ) fails insufficient permissions. similarly, using google contacts api on new target fails 404 errors (for example https://www.googleapis.com/auth/contacts.readonly/default/full ). is there documentation anywhere on how use api, or permissible way access google contacts api full read-write access?
Read more

php - Controller/JToolBar not working in Joomla 2.5 -

relevant code: jtoolbarhelper::custom('savecategories', 'save', '', 'save', false, false); ... <input type="hidden" name="controller" value="easyblogcontroller"> according can dig docs , own previous questions, should call savecategories() function in easyblogcontroller . i've tried setting value easyblog , easyblog.php (file name) current easyblogcontroller (class name). clicking on save button refreshes page. not redirect, echo or var_dump test code put in savecategories() function. var_dump ing jrequest::getvar('controller') , 'task' return correct values. creating controller object , using $controller->execute('task'); works. the values going depend on location of controller trying call, , have couple options here. typically there controller in base folder of component (likely components/com_easyblog) in file called controller.php , class name inside ...
Read more