Agfdhyk

I have created a little script that takes some data from a website table and fills it into cells in Excel. I need this in order to make searching through that data and filtering it easier for myself. However I have ran into a little problem, every time I run my script it does what it's supposed to do for around 9 to 11 pages (about 250 cell entries) and then stops with an error message popping up saying "Runtime Error 91" Object variable or With block variable not set. It never runs all the way through till the end and I only get a small fraction of the data that I was supposed to get.

I am not an expert in programming and even less of an expert in VBA, so if I'm doing something wrong please let me know. I've been stuck on this problem for a few days. I do not know what to do to fix this problem. I've looked for fixes on Google, YouTube and even Microsoft's own website, but can't figure out why this error is popping up in my own script.

I know that this problem is encountered when; 1) you attempt to use an object variable that isn't yet referencing a valid object or when 2) you attempt to use an object variable that has been set to Nothing. But I can't figure in which part of my code I am making that mistake. Help much appreciated.

UPDATE:
When I start the page number from say 8 and it goes up to say 15 it has no problem pulling that data from the site, so I don't think it has anything to do with the specific page numbers (I could be wrong about this). But when I set the page number to 17 instead of 15 the error message pops up almost immediately. So, could it be the range between pages that's causing this?

UPDATE 2:
It's also kind of inconsistent when the error message pops up, when I run the script again with the same page numbers (8 to 17) it runs through the entire thing with no hiccups. (Weird) Then I change the page numbers from (8 to 18) and it gets through 2 pages and stops.

UPDATE 3:
Is it possible that this is happening because I am being rate limited?

Sub GrabInfo()



    Dim objIE As InternetExplorer



    Dim r As Integer

    Dim p As Integer

    Dim c As Integer



    r = 0

    p = 0

    c = 0



    Dim sDD1 As String

    Dim sDD2 As String

    Dim sDD3 As String



    Set objIE = New InternetExplorer

    'objIE.Visible = True



    Do While p < 53

    p = p + 1



        'Setting row to zero after the new page loads but before the data gets pulled

        r = 0



        objIE.navigate "URLGOESHERE.com/123456789?pageNum=" & p

        Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop



        Do While r < 25

        'Increment the row (gets reset after each page)

        r = r + 1

        'Increment the cell (never gets reset)

        c = c + 1



            'Pulling data from site row by row and setting each child to temp sDD strings

            sDD1 = objIE.document.getElementsByClassName("table-content")(r - 1).Children(0).innerText

            sDD2 = objIE.document.getElementsByClassName("table-content")(r - 1).Children(2).innerText

            sDD3 = objIE.document.getElementsByClassName("table-content")(r - 1).Children(3).innerText



            Debug.Print sDD1 & " | " & sDD2 & " | " & sDD3 & " - " & _

            "Cell # = " & c & " ROW # = " & r & " PAGE # = " & p



            'Filling in the temp sDD strings into excel cells

            Range("A" & (c + 1)).Value = sDD1

            Range("B" & (c + 1)).Value = sDD2

            Range("C" & (c + 1)).Value = sDD3



        Loop

    Loop



End Sub