beautifulsoup
Error whilst requesting a login to a site
last EDIT: It seems that i am connecting to the website but for some reason i get an error with my code!Probably i do something wrong with BeautifulSouP.I am thinking that i should change somethin on the url variable! def connect(): with requests.Session() as c: urll2 = "http://www.oddscheck.net/inc/userlogin.php" payload= { 'useremail': 'email#gmail.com', 'userpassword':'password', 'PHPSESSID' : 'ih79c4t5srr6p2', 'CF-RAY' : '35f784ad1-ATH' } headers = {} headers['User-Agent'] = 'Mozilla/5.0 (X11; Linux i686; rv:10.0) Gecko/20100101 Firefox/10.0' c.post(urll2,data=payload,headers={"Referer": "http://www.oddscheck.net/?page=home&cmd=def"}) url = c.get('http://www.oddscheck.net/index.php?page=myaccount') req = urllib.request.Request(url, headers=headers) resp = urllib.request.urlopen(req) soup = BS(resp, "html.parser") gamesave = "" for record in soup.find_all("tr"): game = game1="" for data in record.find_all("td"): if data.get('class') == ['centertd', 'col_10']: for link in data.find_all("a"): game1 += ", "+"http://www.oddscheck.net/"+link.get('href') else: game += ","+data.text game2 = game+game1 if len(game2) > 40: gamesave += game[1:]+","+game1[1:]+"\n" #header = "League,Time,1,X,2,U,O,Link"+"\n" file = open(os.path.expanduser("Odss.txt"),"wb") #file.write(bytes(header, encoding="UTF-8",errors="ignore")) file.write(bytes(gamesave, encoding="UTF-8",errors="ignore")) file.close() ERROR I GET: The error i get: Traceback (most recent call last): File "sportingbet.py", line 314, in <module> connect() File "sportingbet.py", line 27, in connect req = urllib.request.Request(url, headers=headers) File "/usr/lib/python3.5/urllib/request.py", line 269, in __init__ self.full_url = url File "/usr/lib/python3.5/urllib/request.py", line 295, in full_url self._parse() File "/usr/lib/python3.5/urllib/request.py", line 324, in _parse raise ValueError("unknown url type: %r" % self.full_url) ValueError: unknown url type: 'Response [200]
urll = "http://www.oddscheck.net/inc/userlogin.php" I'd take the extra 'l' out in urll
The client-side variable name for your URL must be consistent with that of the variable for the server-side. Assuming you already know the name of it, you should just be able to change it to that and have it run.
Related Links
Error “global name 'BeautifulSoup' is not defined”
beautifulsoup get tags after <p> tag after <h3> and <br> tag between <p>
AttributeError: ' ' object has no attribute 'soup'
How to scrap whole website using beautifulsoup
BeautifulSoup Nested class selector
Beautiful Soup get the second attribute from element
beautiful soup parse a tags with inconsistent hrefs
How to crawl a website that uses dot.js using beautifulsoup?
Python BeautifulSoup how to extract/find
HOW DO I PARSE THE DATA IN THESE HTML TAGS?
BeautifulSoup's find method returns Tag item, but call string get 'NoneType' Error
Why is Beautifulsoup find_all not returning complete results?
How to iterate the tags under same tag?
How to scrape a tag under javascript tag?
Find productID with beautifulsoup
python: beautifulsoup4: scrapping: Unable to figure out the Error