Selenium 101 – where to start to understand web page scraping automation

I am using nodejs as my language, but the examples I was given to hack and understand our project always loaded the Selenium Webdriver. What is this beast? Here is a great starter article to get going. Here is the real McCoy but harder to understand. If you want a click-and-record version like in Excel-VBA – Selenium IDE which is a plug in for Firefox and might be a great option.

After playing for a little – there was no ‘export as javascript’ to be as close to nodejs as possible – but after saying “really – there is no export for javascript!?” I went to good and I did find two. The first one worked so I didn’t pursue the 2nd option. Both likely work.

• selenium-ide-webdriver-node (literally – just do his install steps – had to disable the ‘signed code security’ in the about:config part of firefox by setting the xpinstall.signatures.required=false
• js-wd-formatter

I also installed a php one from the firefox addins page by searching for “selenium php exporter” or “selenium php formatter”.