0

I want to read 'title' of list of projects which are in pagination & 335 records almost. What i am trying to do is : 1) First I get response of the browser by this command in windows cmd:

scrapy shell https://www.slingshotinsights.com/projects

2)It shows the HTML rendered in cmd, and very next i write

reponse.css('a.grey-link').extract()

and press enter, it gives me [] 'empty array'.

The question is, How to get data from scrapy script for the URL's which appears after login? Because https://www.slingshotinsights.com/projects is the link, which comes when user successfully login the page.

And may be scrapy is unable to find that 

reponse.css('a.grey-link').extract()

css selector becuase it can't be loaded in logout view.

AutomationEngineer
  • 29
  • 5
  • You have to login to that website if website is displaying data for logged in user only ... – Umair Ayub Nov 02 '17 at 11:18
  • Possible duplicate of [Using Scrapy with authenticated (logged in) user session](https://stackoverflow.com/questions/5850755/using-scrapy-with-authenticated-logged-in-user-session) – Umair Ayub Nov 02 '17 at 11:19

0 Answers0