Web Scraping with urllib in Python 3

Here is an example of logging in to some website, and get some content.

#!/usr/bin/python3
# Importing modules for handling http and cookie
import http.cookiejar, urllib.request

# Storing cookies in cj variable
cj = http.cookiejar.CookieJar()

# Defining a handler for later http operations with cookies(cj).
op = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))

# Logging in
url = ('https://127.0.0.1/index.php?')
val = {'user' : 'username', 'password' : 'password'}
data = urllib.parse.urlencode(val)
asciidata = data.encode('ascii')
res = opener.open(url, asciidata)

# Saving a file
f = open("content.jpg", "wb")
res = op.open('https://127.0.0.1/index.php/apps/contents.jpg')
f.write(res.read())
f.close()

Advertisements
Leave a comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: