Looping over a Table Extraction URL?

[RFear]

Hello,

I am trying to extract tables which are on several different pages of a website, hence I have made lots of 'Extract Table' commands and want to loop over the URL which is at the top of the table extraction commands.

This isn't working though as I end up with .csv's for all the entries of the variable/loop but all of them contain data which is from only the first element of the variable/loop. Basically the URL is not looping.

Is there another way to do this?

I have attached a screen shot of my attempt.

Thanks,
Richard.

[RFear]

Forgot the attachment!

[trentonponder]

Originally Posted by RFear

Hello,

I am trying to extract tables which are on several different pages of a website, hence I have made lots of 'Extract Table' commands and want to loop over the URL which is at the top of the table extraction commands.

This isn't working though as I end up with .csv's for all the entries of the variable/loop but all of them contain data which is from only the first element of the variable/loop. Basically the URL is not looping.

Is there another way to do this?

I have attached a screen shot of my attempt.

Thanks,
Richard.

Your image is too small and I don't quite understand the issue.

[RFear]

Ok, try this: https://docs.google.com/open?id=0Bw1...hhNWFiYzJiODEz

The idea is that I want to extract CIA world factbook data on all the countries of the world.

Hence I have made several table extraction commands which work fine for scraping the data I need on a single country.

However when I try to do this for the other countries by putting a variable into the URL which determines where the tables will be extracted from it does not work, I just get the data for the first country extracted multiple times.

($cc$ is the variable which has entries like 'as' for the Australian page, 'af' for the Afghanistani page etc...).

Does that clear it up?

[trentonponder]

Originally Posted by RFear

Ok, try this: https://docs.google.com/open?id=0Bw1...hhNWFiYzJiODEz

The idea is that I want to extract CIA world factbook data on all the countries of the world.

Hence I have made several table extraction commands which work fine for scraping the data I need on a single country.

However when I try to do this for the other countries by putting a variable into the URL which determines where the tables will be extracted from it does not work, I just get the data for the first country extracted multiple times.

($cc$ is the variable which has entries like 'as' for the Australian page, 'af' for the Afghanistani page etc...).

Does that clear it up?

Much better image. You don't need to have multiple extractions in your loop, you only need it once. So, your code should look like this:

Start Loop "List Variable cc"

Extract from https://www.cia.gov/library/publications/the-world- factbook/geos/countrytemplate_$cc$.html

End Loop


The loop will replace the CC each time with the values in the list variable. The way you're doing it now, it keeps on using the same variable 9 times (although if you let the loop run it will be each country 9 times)

Hope that helps.

[RFear]

Thanks for your answer.

The reason I have 9 extraction commands is that there are actually 9 tables on each page which I want to extract.

Other than that I expected it to work as you describe but instead I get the same page (whatever the first page in the variable list is) extracted over and over again.

Any idea what I am doing wrong?

[forumstaff]

Hi,

Can you attach a copy of the task that you have created at your end. To locate the task file, right click on it and select the 'Locate on Disk' option.

The task script will give us a better understanding and help us in reaching a possible solution.

Regards.

[forumstaff]

Hello RFear,

Based on the task file you sent us to support, we have modified and prepared the sample task for you.

Now, it is getting data from different URLS that you are looking to be extracted in different CSV file. Also, you do not have to go and capture each table manually from webpage, task will get data using one Extract Table command only.

Please save the attached task at My Documents\Automation Anywhere\Automation Anywhere\My Tasks. Launch Automation Anywhere, select the task and click on Run.

We have made several changes in the task. However few important changes you can notice here is as follows:
1. Task needs to be navigated to the URL first from where data is needed to be Extracted, earlier it was not the case.
2. In Extract Table command there is option to continue with the task even if the command Error out. This allowed to loop through each table number present on website and if the table number is not there it will error out and move to next table number.
3. Task runs on two variables one which will loop through each links formation and second counter variable takes care of looping through each Table.

Let us know how does it goes for you.