Processing time to compare data took more than an hour in VBA











up vote
-1
down vote

favorite












I've been doing a tool which is to compare 2 workbooks. I don't have any problem in my code, is just that, if I have a maximum rows it took me more than 5hrs to process the data.



I need to compare the previous and latter file, if the previous file is not in the latter file then the rows are deleted. While, If I compare the latter to previous and there are rows that have been added in the latter file then the rows are added. The Data contains delimiter "¦" (e.g. 1315083¦Tamayo¦Juan¦Juan.Tamayo@xyz.com¦admin¦Requestor). When I split the data into "¦", the 1st and last array (1315083 and Requestor) will be used to compare the data. I used LCase in may condition to accept any casing.



Dim wbCopyT As Workbook
Set wbCopyT = Workbooks.Add

Dim arr As Variant
arr = wbCopyT.Sheets("Sheet3").Range("A1:A" & wbCopyT.Sheets("Sheet3").Range("A" & Rows.count).End(xlUp).Row).Value

Dim varr As Variant
varr = wbCopyT.Sheets("Sheet3").Range("B1:B" & wbCopyT.Sheets("Sheet3").Range("B" & Rows.count).End(xlUp).Row).Value

'compare data from previous to latter file
Dim x As Variant, y As Variant, match As Boolean
Dim splitVal1 As Variant, splitVal2 As Variant
Dim output1 As String, output2 As String

For Each x In arr
match = False
splitVal1 = Split(x, "¦")
output1 = LCase(splitVal1(0)) & LCase(splitVal1(5))
For Each y In varr
splitVal2 = Split(y, "¦")
output2 = LCase(splitVal2(0)) & LCase(splitVal2(5))
If output1 = output2 Then
wbCopyT.Sheets("Sheet3").Range("C" & wbCopyT.Sheets("Sheet3").Range("C" & Rows.count).End(xlUp).Row + 1) = x & "~No update/change"
match = True
End If
Next y
If Not match Then
wbCopyT.Sheets("Sheet3").Range("C" & wbCopyT.Sheets("Sheet3").Range("C" & Rows.count).End(xlUp).Row + 1) = x & "~Deleted"
End If
Next x

'compare data from latter to previous file
Dim a As Variant, b As Variant, match2 As Boolean
Dim splitVal3 As Variant, splitVal4 As Variant
Dim output3 As String, output4 As String

For Each a In varr
match2 = False
splitVal3 = Split(a, "¦")
output3 = LCase(splitVal3(0)) & LCase(splitVal3(5))
For Each b In arr
splitVal4 = Split(b, "¦")
output4 = LCase(splitVal4(0)) & LCase(splitVal4(5))
If output3 = output4 Then
wbCopyT.Sheets("Sheet3").Range("D" & wbCopyT.Sheets("Sheet3").Range("D" & Rows.count).End(xlUp).Row + 1) = a & "~No update/change"
match2 = True
End If
Next b
If Not match2 Then
wbCopyT.Sheets("Sheet3").Range("D" & wbCopyT.Sheets("Sheet3").Range("D" & Rows.count).End(xlUp).Row + 1) = a & "~Added"
End If
Next a


I need to limit the processing time but I don't know how to do that. Is there a problem with my code? THank you.










share|improve this question






















  • Add the line Debug.Print wbCopyT.Sheets("Sheet3").Range("A" & Rows.Count).End(xlUp).Row).Value and Debug.Print wbCopyT.Sheets("Sheet3").Range("B" & Rows.Count).End(xlUp).Row).Value right about your "compare data from previous to latter file". Are you sure you're not looping through a million rows? See what those values are in the immediate window.
    – dwirony
    Nov 9 at 15:18












  • How does this code even run? You're adding a new workbook, then immediately taking a lastrow value from a sheet that doesn't exist. You should get a run-time error 9 on your arr = line. What am I missing here?
    – dwirony
    Nov 9 at 15:24










  • You're doing way too much in the inner loops. You need to cache the results of the Split operations. For example, splitVal2 = Split(y, "¦") is always going to be the same value for any value of For Each x In arr.
    – Comintern
    Nov 9 at 15:26










  • @dwirony - i am looping for about 600k rows. wbCopyT is another workbook, after I copied all the data from the 2 other workbook. I need to paste it in this workbook wbCopyT .
    – Yoona May
    Nov 9 at 15:45










  • @dwirony - got an error when I add the Debug.Print. It says "Type Mismatch".
    – Yoona May
    Nov 9 at 15:46















up vote
-1
down vote

favorite












I've been doing a tool which is to compare 2 workbooks. I don't have any problem in my code, is just that, if I have a maximum rows it took me more than 5hrs to process the data.



I need to compare the previous and latter file, if the previous file is not in the latter file then the rows are deleted. While, If I compare the latter to previous and there are rows that have been added in the latter file then the rows are added. The Data contains delimiter "¦" (e.g. 1315083¦Tamayo¦Juan¦Juan.Tamayo@xyz.com¦admin¦Requestor). When I split the data into "¦", the 1st and last array (1315083 and Requestor) will be used to compare the data. I used LCase in may condition to accept any casing.



Dim wbCopyT As Workbook
Set wbCopyT = Workbooks.Add

Dim arr As Variant
arr = wbCopyT.Sheets("Sheet3").Range("A1:A" & wbCopyT.Sheets("Sheet3").Range("A" & Rows.count).End(xlUp).Row).Value

Dim varr As Variant
varr = wbCopyT.Sheets("Sheet3").Range("B1:B" & wbCopyT.Sheets("Sheet3").Range("B" & Rows.count).End(xlUp).Row).Value

'compare data from previous to latter file
Dim x As Variant, y As Variant, match As Boolean
Dim splitVal1 As Variant, splitVal2 As Variant
Dim output1 As String, output2 As String

For Each x In arr
match = False
splitVal1 = Split(x, "¦")
output1 = LCase(splitVal1(0)) & LCase(splitVal1(5))
For Each y In varr
splitVal2 = Split(y, "¦")
output2 = LCase(splitVal2(0)) & LCase(splitVal2(5))
If output1 = output2 Then
wbCopyT.Sheets("Sheet3").Range("C" & wbCopyT.Sheets("Sheet3").Range("C" & Rows.count).End(xlUp).Row + 1) = x & "~No update/change"
match = True
End If
Next y
If Not match Then
wbCopyT.Sheets("Sheet3").Range("C" & wbCopyT.Sheets("Sheet3").Range("C" & Rows.count).End(xlUp).Row + 1) = x & "~Deleted"
End If
Next x

'compare data from latter to previous file
Dim a As Variant, b As Variant, match2 As Boolean
Dim splitVal3 As Variant, splitVal4 As Variant
Dim output3 As String, output4 As String

For Each a In varr
match2 = False
splitVal3 = Split(a, "¦")
output3 = LCase(splitVal3(0)) & LCase(splitVal3(5))
For Each b In arr
splitVal4 = Split(b, "¦")
output4 = LCase(splitVal4(0)) & LCase(splitVal4(5))
If output3 = output4 Then
wbCopyT.Sheets("Sheet3").Range("D" & wbCopyT.Sheets("Sheet3").Range("D" & Rows.count).End(xlUp).Row + 1) = a & "~No update/change"
match2 = True
End If
Next b
If Not match2 Then
wbCopyT.Sheets("Sheet3").Range("D" & wbCopyT.Sheets("Sheet3").Range("D" & Rows.count).End(xlUp).Row + 1) = a & "~Added"
End If
Next a


I need to limit the processing time but I don't know how to do that. Is there a problem with my code? THank you.










share|improve this question






















  • Add the line Debug.Print wbCopyT.Sheets("Sheet3").Range("A" & Rows.Count).End(xlUp).Row).Value and Debug.Print wbCopyT.Sheets("Sheet3").Range("B" & Rows.Count).End(xlUp).Row).Value right about your "compare data from previous to latter file". Are you sure you're not looping through a million rows? See what those values are in the immediate window.
    – dwirony
    Nov 9 at 15:18












  • How does this code even run? You're adding a new workbook, then immediately taking a lastrow value from a sheet that doesn't exist. You should get a run-time error 9 on your arr = line. What am I missing here?
    – dwirony
    Nov 9 at 15:24










  • You're doing way too much in the inner loops. You need to cache the results of the Split operations. For example, splitVal2 = Split(y, "¦") is always going to be the same value for any value of For Each x In arr.
    – Comintern
    Nov 9 at 15:26










  • @dwirony - i am looping for about 600k rows. wbCopyT is another workbook, after I copied all the data from the 2 other workbook. I need to paste it in this workbook wbCopyT .
    – Yoona May
    Nov 9 at 15:45










  • @dwirony - got an error when I add the Debug.Print. It says "Type Mismatch".
    – Yoona May
    Nov 9 at 15:46













up vote
-1
down vote

favorite









up vote
-1
down vote

favorite











I've been doing a tool which is to compare 2 workbooks. I don't have any problem in my code, is just that, if I have a maximum rows it took me more than 5hrs to process the data.



I need to compare the previous and latter file, if the previous file is not in the latter file then the rows are deleted. While, If I compare the latter to previous and there are rows that have been added in the latter file then the rows are added. The Data contains delimiter "¦" (e.g. 1315083¦Tamayo¦Juan¦Juan.Tamayo@xyz.com¦admin¦Requestor). When I split the data into "¦", the 1st and last array (1315083 and Requestor) will be used to compare the data. I used LCase in may condition to accept any casing.



Dim wbCopyT As Workbook
Set wbCopyT = Workbooks.Add

Dim arr As Variant
arr = wbCopyT.Sheets("Sheet3").Range("A1:A" & wbCopyT.Sheets("Sheet3").Range("A" & Rows.count).End(xlUp).Row).Value

Dim varr As Variant
varr = wbCopyT.Sheets("Sheet3").Range("B1:B" & wbCopyT.Sheets("Sheet3").Range("B" & Rows.count).End(xlUp).Row).Value

'compare data from previous to latter file
Dim x As Variant, y As Variant, match As Boolean
Dim splitVal1 As Variant, splitVal2 As Variant
Dim output1 As String, output2 As String

For Each x In arr
match = False
splitVal1 = Split(x, "¦")
output1 = LCase(splitVal1(0)) & LCase(splitVal1(5))
For Each y In varr
splitVal2 = Split(y, "¦")
output2 = LCase(splitVal2(0)) & LCase(splitVal2(5))
If output1 = output2 Then
wbCopyT.Sheets("Sheet3").Range("C" & wbCopyT.Sheets("Sheet3").Range("C" & Rows.count).End(xlUp).Row + 1) = x & "~No update/change"
match = True
End If
Next y
If Not match Then
wbCopyT.Sheets("Sheet3").Range("C" & wbCopyT.Sheets("Sheet3").Range("C" & Rows.count).End(xlUp).Row + 1) = x & "~Deleted"
End If
Next x

'compare data from latter to previous file
Dim a As Variant, b As Variant, match2 As Boolean
Dim splitVal3 As Variant, splitVal4 As Variant
Dim output3 As String, output4 As String

For Each a In varr
match2 = False
splitVal3 = Split(a, "¦")
output3 = LCase(splitVal3(0)) & LCase(splitVal3(5))
For Each b In arr
splitVal4 = Split(b, "¦")
output4 = LCase(splitVal4(0)) & LCase(splitVal4(5))
If output3 = output4 Then
wbCopyT.Sheets("Sheet3").Range("D" & wbCopyT.Sheets("Sheet3").Range("D" & Rows.count).End(xlUp).Row + 1) = a & "~No update/change"
match2 = True
End If
Next b
If Not match2 Then
wbCopyT.Sheets("Sheet3").Range("D" & wbCopyT.Sheets("Sheet3").Range("D" & Rows.count).End(xlUp).Row + 1) = a & "~Added"
End If
Next a


I need to limit the processing time but I don't know how to do that. Is there a problem with my code? THank you.










share|improve this question













I've been doing a tool which is to compare 2 workbooks. I don't have any problem in my code, is just that, if I have a maximum rows it took me more than 5hrs to process the data.



I need to compare the previous and latter file, if the previous file is not in the latter file then the rows are deleted. While, If I compare the latter to previous and there are rows that have been added in the latter file then the rows are added. The Data contains delimiter "¦" (e.g. 1315083¦Tamayo¦Juan¦Juan.Tamayo@xyz.com¦admin¦Requestor). When I split the data into "¦", the 1st and last array (1315083 and Requestor) will be used to compare the data. I used LCase in may condition to accept any casing.



Dim wbCopyT As Workbook
Set wbCopyT = Workbooks.Add

Dim arr As Variant
arr = wbCopyT.Sheets("Sheet3").Range("A1:A" & wbCopyT.Sheets("Sheet3").Range("A" & Rows.count).End(xlUp).Row).Value

Dim varr As Variant
varr = wbCopyT.Sheets("Sheet3").Range("B1:B" & wbCopyT.Sheets("Sheet3").Range("B" & Rows.count).End(xlUp).Row).Value

'compare data from previous to latter file
Dim x As Variant, y As Variant, match As Boolean
Dim splitVal1 As Variant, splitVal2 As Variant
Dim output1 As String, output2 As String

For Each x In arr
match = False
splitVal1 = Split(x, "¦")
output1 = LCase(splitVal1(0)) & LCase(splitVal1(5))
For Each y In varr
splitVal2 = Split(y, "¦")
output2 = LCase(splitVal2(0)) & LCase(splitVal2(5))
If output1 = output2 Then
wbCopyT.Sheets("Sheet3").Range("C" & wbCopyT.Sheets("Sheet3").Range("C" & Rows.count).End(xlUp).Row + 1) = x & "~No update/change"
match = True
End If
Next y
If Not match Then
wbCopyT.Sheets("Sheet3").Range("C" & wbCopyT.Sheets("Sheet3").Range("C" & Rows.count).End(xlUp).Row + 1) = x & "~Deleted"
End If
Next x

'compare data from latter to previous file
Dim a As Variant, b As Variant, match2 As Boolean
Dim splitVal3 As Variant, splitVal4 As Variant
Dim output3 As String, output4 As String

For Each a In varr
match2 = False
splitVal3 = Split(a, "¦")
output3 = LCase(splitVal3(0)) & LCase(splitVal3(5))
For Each b In arr
splitVal4 = Split(b, "¦")
output4 = LCase(splitVal4(0)) & LCase(splitVal4(5))
If output3 = output4 Then
wbCopyT.Sheets("Sheet3").Range("D" & wbCopyT.Sheets("Sheet3").Range("D" & Rows.count).End(xlUp).Row + 1) = a & "~No update/change"
match2 = True
End If
Next b
If Not match2 Then
wbCopyT.Sheets("Sheet3").Range("D" & wbCopyT.Sheets("Sheet3").Range("D" & Rows.count).End(xlUp).Row + 1) = a & "~Added"
End If
Next a


I need to limit the processing time but I don't know how to do that. Is there a problem with my code? THank you.







excel vba excel-vba






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 9 at 15:13









Yoona May

517




517












  • Add the line Debug.Print wbCopyT.Sheets("Sheet3").Range("A" & Rows.Count).End(xlUp).Row).Value and Debug.Print wbCopyT.Sheets("Sheet3").Range("B" & Rows.Count).End(xlUp).Row).Value right about your "compare data from previous to latter file". Are you sure you're not looping through a million rows? See what those values are in the immediate window.
    – dwirony
    Nov 9 at 15:18












  • How does this code even run? You're adding a new workbook, then immediately taking a lastrow value from a sheet that doesn't exist. You should get a run-time error 9 on your arr = line. What am I missing here?
    – dwirony
    Nov 9 at 15:24










  • You're doing way too much in the inner loops. You need to cache the results of the Split operations. For example, splitVal2 = Split(y, "¦") is always going to be the same value for any value of For Each x In arr.
    – Comintern
    Nov 9 at 15:26










  • @dwirony - i am looping for about 600k rows. wbCopyT is another workbook, after I copied all the data from the 2 other workbook. I need to paste it in this workbook wbCopyT .
    – Yoona May
    Nov 9 at 15:45










  • @dwirony - got an error when I add the Debug.Print. It says "Type Mismatch".
    – Yoona May
    Nov 9 at 15:46


















  • Add the line Debug.Print wbCopyT.Sheets("Sheet3").Range("A" & Rows.Count).End(xlUp).Row).Value and Debug.Print wbCopyT.Sheets("Sheet3").Range("B" & Rows.Count).End(xlUp).Row).Value right about your "compare data from previous to latter file". Are you sure you're not looping through a million rows? See what those values are in the immediate window.
    – dwirony
    Nov 9 at 15:18












  • How does this code even run? You're adding a new workbook, then immediately taking a lastrow value from a sheet that doesn't exist. You should get a run-time error 9 on your arr = line. What am I missing here?
    – dwirony
    Nov 9 at 15:24










  • You're doing way too much in the inner loops. You need to cache the results of the Split operations. For example, splitVal2 = Split(y, "¦") is always going to be the same value for any value of For Each x In arr.
    – Comintern
    Nov 9 at 15:26










  • @dwirony - i am looping for about 600k rows. wbCopyT is another workbook, after I copied all the data from the 2 other workbook. I need to paste it in this workbook wbCopyT .
    – Yoona May
    Nov 9 at 15:45










  • @dwirony - got an error when I add the Debug.Print. It says "Type Mismatch".
    – Yoona May
    Nov 9 at 15:46
















Add the line Debug.Print wbCopyT.Sheets("Sheet3").Range("A" & Rows.Count).End(xlUp).Row).Value and Debug.Print wbCopyT.Sheets("Sheet3").Range("B" & Rows.Count).End(xlUp).Row).Value right about your "compare data from previous to latter file". Are you sure you're not looping through a million rows? See what those values are in the immediate window.
– dwirony
Nov 9 at 15:18






Add the line Debug.Print wbCopyT.Sheets("Sheet3").Range("A" & Rows.Count).End(xlUp).Row).Value and Debug.Print wbCopyT.Sheets("Sheet3").Range("B" & Rows.Count).End(xlUp).Row).Value right about your "compare data from previous to latter file". Are you sure you're not looping through a million rows? See what those values are in the immediate window.
– dwirony
Nov 9 at 15:18














How does this code even run? You're adding a new workbook, then immediately taking a lastrow value from a sheet that doesn't exist. You should get a run-time error 9 on your arr = line. What am I missing here?
– dwirony
Nov 9 at 15:24




How does this code even run? You're adding a new workbook, then immediately taking a lastrow value from a sheet that doesn't exist. You should get a run-time error 9 on your arr = line. What am I missing here?
– dwirony
Nov 9 at 15:24












You're doing way too much in the inner loops. You need to cache the results of the Split operations. For example, splitVal2 = Split(y, "¦") is always going to be the same value for any value of For Each x In arr.
– Comintern
Nov 9 at 15:26




You're doing way too much in the inner loops. You need to cache the results of the Split operations. For example, splitVal2 = Split(y, "¦") is always going to be the same value for any value of For Each x In arr.
– Comintern
Nov 9 at 15:26












@dwirony - i am looping for about 600k rows. wbCopyT is another workbook, after I copied all the data from the 2 other workbook. I need to paste it in this workbook wbCopyT .
– Yoona May
Nov 9 at 15:45




@dwirony - i am looping for about 600k rows. wbCopyT is another workbook, after I copied all the data from the 2 other workbook. I need to paste it in this workbook wbCopyT .
– Yoona May
Nov 9 at 15:45












@dwirony - got an error when I add the Debug.Print. It says "Type Mismatch".
– Yoona May
Nov 9 at 15:46




@dwirony - got an error when I add the Debug.Print. It says "Type Mismatch".
– Yoona May
Nov 9 at 15:46

















active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














 

draft saved


draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53228399%2fprocessing-time-to-compare-data-took-more-than-an-hour-in-vba%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown






























active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes
















 

draft saved


draft discarded



















































 


draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53228399%2fprocessing-time-to-compare-data-took-more-than-an-hour-in-vba%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Guess what letter conforming each word

Port of Spain

Run scheduled task as local user group (not BUILTIN)