javascript - How to scroll down with Phantomjs to load dynamic content -

i trying scrape links page generates content dynamically user scroll down bottom (infinite scrolling). have tried doing different things phantomjs not able gather links beyond first page. let element @ bottom loads content has class .has-more-items. available until final content loaded while scrolling , becomes unavailable in dom (display:none). here things have tried-

• setting viewportsize large height right after var page = require('webpage').create();

page.viewportsize = { width: 1600, height: 10000, };

• using page.scrollposition = { top: 10000, left: 0 } inside page.open have no effect like-

page.open('http://example.com/?q=houston', function(status) {    if (status == "success") {       page.scrollposition = { top: 10000, left: 0 };      } }); 

• also tried putting inside page.evaluate function gives

reference error: can't find variable page

• tried using jquery , js code inside page.evaluate , page.open no avail-

$("html, body").animate({ scrolltop: $(document).height() }, 10, function() { //console.log('check execution'); });

as , inside document.ready. js code-

window.scrollby(0,10000) 

as , inside window.onload

i struck on 2 days , not able find way. or hint appreciated.

update

i have found helpful piece of code @ https://groups.google.com/forum/?fromgroups=#!topic/phantomjs/8lrwrw8zra0

var hitrockbottom = false; while (!hitrockbottom) {     // scroll page (not sure if best way so...)     page.scrollposition = { top: page.scrollposition + 1000, left: 0 };      // check if we've hit bottom     hitrockbottom = page.evaluate(function() {         return document.queryselector(".has-more-items") === null;     }); } 

where .has-more-items element class want access available @ bottom of page , scroll down, moves further down until data loaded , becomes unavailable.

however, when tested clear running infinite loops without scrolling down (i render pictures check). have tried replace page.scrollposition = { top: page.scrollposition + 1000, left: 0 }; codes below (one @ time)

window.document.body.scrolltop = '1000'; location.href = ".has-more-items"; page.scrollposition = { top: page.scrollposition + 1000, left: 0 }; document.location.href=".has-more-items"; 

but nothing seems work.

found way , tried adapt situation. didn't test best way of finding bottom of page because had different context, check out. problem have wait little page load out , javascript works asynchronously have use setinterval or settimeout (see).

page.open('http://example.com/?q=houston', function () {    // checks bottom div , scrolls down time time   window.setinterval(function() {       // checks if there div class=".has-more-items"        // (not sure if best way of doing it)       var count = page.content.match(/class=".has-more-items"/g);        if(count === null) { // didn't find         page.evaluate(function() {           // scrolls bottom of page           window.document.body.scrolltop = document.body.scrollheight;         });       }       else { // found         // want         ...         phantom.exit();       }   }, 500); // number of milliseconds wait between scrolls  });