Inconsistent query execution time











up vote
0
down vote

favorite












I have the same query executing over and over and sometimes it is ultra quick, other times it takes much longer. Any one know what could be causing this?



I have a Content Provider that is accessed via a loader on my activity.



It has to filter through 10,000 plus entries into a database and I had optimized it significantly to reduce query time.



Often the query runs in 0-30 ms. As I print out info every time the query executes.



But I have noticed now with more data, sometimes the query will run super quick, and other times it will take 3-5 seconds. Using the same selection/query/arguments as the fast execution.



Log is printing out the times below: And the query is exactly the same.



Query took 15ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 15ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 3499ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 3502ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 18ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 6ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 3618ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 3601ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]


I record this above log as such in the Provider



queryBuilder.setProjectionMap(projectionMap);
long start = System.currentTimeMillis();
retCursor = queryBuilder.query(mOpenHelper.getReadableDatabase(), null,
selection,
selectionArgs,
null,
null,
null, null);
//This goes with the log statement, to prevent a null pointer on selectionArgs
if (selectionArgs == null) {
selectionArgs = new String{"null"};
}
Log.d(LOG_TAG, "Query took " + (System.currentTimeMillis() - start) + "ms to complete SELECTION:" +
" Rows: " + retCursor.getCount() +
" :: " + selection + " sel args: " + Arrays.toString(selectionArgs));


Edit: The table I am pulling from has half a million rows. But I put a column in with a flag to indicate if it was the last record inserted for a specific user. And indexed it. This allowed me to get a row for each users last position really quickly. A new position removes the flag on the old position and inserts the new position with the flag set. I can query the table in less than 20ms, or the ones that seem to get stuck at about 2.5 seconds or more. which seems to happen about 30-40% (est) of the time now.










share|improve this question




















  • 1




    Have you tried running the same query in a database tool? Does it run in constant time there?
    – Danail Alexiev
    Nov 8 at 21:05










  • Is there a recommended tool for testing in such a manner on a phone? Ill have to extract the query from the querybuilder I am using.
    – Charon
    Nov 8 at 21:07






  • 1




    You can extract your database file and open it via sqlitebrowser.org. Then you can run the query and check the execution times.
    – Danail Alexiev
    Nov 8 at 22:01










  • I extracted the DB file and the query using the debugger. And on my computer the query runs in 0ms, returns 3 rows. I should note the table actually has half a million rows.
    – Charon
    Nov 9 at 1:12












  • Is there a difference in the statements / parameters that cause the speed to drop or they are just the same as the quick queries?
    – Danail Alexiev
    Nov 9 at 10:19















up vote
0
down vote

favorite












I have the same query executing over and over and sometimes it is ultra quick, other times it takes much longer. Any one know what could be causing this?



I have a Content Provider that is accessed via a loader on my activity.



It has to filter through 10,000 plus entries into a database and I had optimized it significantly to reduce query time.



Often the query runs in 0-30 ms. As I print out info every time the query executes.



But I have noticed now with more data, sometimes the query will run super quick, and other times it will take 3-5 seconds. Using the same selection/query/arguments as the fast execution.



Log is printing out the times below: And the query is exactly the same.



Query took 15ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 15ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 3499ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 3502ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 18ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 6ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 3618ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 3601ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]


I record this above log as such in the Provider



queryBuilder.setProjectionMap(projectionMap);
long start = System.currentTimeMillis();
retCursor = queryBuilder.query(mOpenHelper.getReadableDatabase(), null,
selection,
selectionArgs,
null,
null,
null, null);
//This goes with the log statement, to prevent a null pointer on selectionArgs
if (selectionArgs == null) {
selectionArgs = new String{"null"};
}
Log.d(LOG_TAG, "Query took " + (System.currentTimeMillis() - start) + "ms to complete SELECTION:" +
" Rows: " + retCursor.getCount() +
" :: " + selection + " sel args: " + Arrays.toString(selectionArgs));


Edit: The table I am pulling from has half a million rows. But I put a column in with a flag to indicate if it was the last record inserted for a specific user. And indexed it. This allowed me to get a row for each users last position really quickly. A new position removes the flag on the old position and inserts the new position with the flag set. I can query the table in less than 20ms, or the ones that seem to get stuck at about 2.5 seconds or more. which seems to happen about 30-40% (est) of the time now.










share|improve this question




















  • 1




    Have you tried running the same query in a database tool? Does it run in constant time there?
    – Danail Alexiev
    Nov 8 at 21:05










  • Is there a recommended tool for testing in such a manner on a phone? Ill have to extract the query from the querybuilder I am using.
    – Charon
    Nov 8 at 21:07






  • 1




    You can extract your database file and open it via sqlitebrowser.org. Then you can run the query and check the execution times.
    – Danail Alexiev
    Nov 8 at 22:01










  • I extracted the DB file and the query using the debugger. And on my computer the query runs in 0ms, returns 3 rows. I should note the table actually has half a million rows.
    – Charon
    Nov 9 at 1:12












  • Is there a difference in the statements / parameters that cause the speed to drop or they are just the same as the quick queries?
    – Danail Alexiev
    Nov 9 at 10:19













up vote
0
down vote

favorite









up vote
0
down vote

favorite











I have the same query executing over and over and sometimes it is ultra quick, other times it takes much longer. Any one know what could be causing this?



I have a Content Provider that is accessed via a loader on my activity.



It has to filter through 10,000 plus entries into a database and I had optimized it significantly to reduce query time.



Often the query runs in 0-30 ms. As I print out info every time the query executes.



But I have noticed now with more data, sometimes the query will run super quick, and other times it will take 3-5 seconds. Using the same selection/query/arguments as the fast execution.



Log is printing out the times below: And the query is exactly the same.



Query took 15ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 15ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 3499ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 3502ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 18ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 6ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 3618ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 3601ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]


I record this above log as such in the Provider



queryBuilder.setProjectionMap(projectionMap);
long start = System.currentTimeMillis();
retCursor = queryBuilder.query(mOpenHelper.getReadableDatabase(), null,
selection,
selectionArgs,
null,
null,
null, null);
//This goes with the log statement, to prevent a null pointer on selectionArgs
if (selectionArgs == null) {
selectionArgs = new String{"null"};
}
Log.d(LOG_TAG, "Query took " + (System.currentTimeMillis() - start) + "ms to complete SELECTION:" +
" Rows: " + retCursor.getCount() +
" :: " + selection + " sel args: " + Arrays.toString(selectionArgs));


Edit: The table I am pulling from has half a million rows. But I put a column in with a flag to indicate if it was the last record inserted for a specific user. And indexed it. This allowed me to get a row for each users last position really quickly. A new position removes the flag on the old position and inserts the new position with the flag set. I can query the table in less than 20ms, or the ones that seem to get stuck at about 2.5 seconds or more. which seems to happen about 30-40% (est) of the time now.










share|improve this question















I have the same query executing over and over and sometimes it is ultra quick, other times it takes much longer. Any one know what could be causing this?



I have a Content Provider that is accessed via a loader on my activity.



It has to filter through 10,000 plus entries into a database and I had optimized it significantly to reduce query time.



Often the query runs in 0-30 ms. As I print out info every time the query executes.



But I have noticed now with more data, sometimes the query will run super quick, and other times it will take 3-5 seconds. Using the same selection/query/arguments as the fast execution.



Log is printing out the times below: And the query is exactly the same.



Query took 15ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 15ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 3499ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 3502ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 18ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 6ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 3618ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]
Query took 3601ms to complete SELECTION: Rows: 3 :: type != ? sel args: [0]


I record this above log as such in the Provider



queryBuilder.setProjectionMap(projectionMap);
long start = System.currentTimeMillis();
retCursor = queryBuilder.query(mOpenHelper.getReadableDatabase(), null,
selection,
selectionArgs,
null,
null,
null, null);
//This goes with the log statement, to prevent a null pointer on selectionArgs
if (selectionArgs == null) {
selectionArgs = new String{"null"};
}
Log.d(LOG_TAG, "Query took " + (System.currentTimeMillis() - start) + "ms to complete SELECTION:" +
" Rows: " + retCursor.getCount() +
" :: " + selection + " sel args: " + Arrays.toString(selectionArgs));


Edit: The table I am pulling from has half a million rows. But I put a column in with a flag to indicate if it was the last record inserted for a specific user. And indexed it. This allowed me to get a row for each users last position really quickly. A new position removes the flag on the old position and inserts the new position with the flag set. I can query the table in less than 20ms, or the ones that seem to get stuck at about 2.5 seconds or more. which seems to happen about 30-40% (est) of the time now.







android sqlite loader






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 9 at 1:17

























asked Nov 8 at 20:52









Charon

3211




3211








  • 1




    Have you tried running the same query in a database tool? Does it run in constant time there?
    – Danail Alexiev
    Nov 8 at 21:05










  • Is there a recommended tool for testing in such a manner on a phone? Ill have to extract the query from the querybuilder I am using.
    – Charon
    Nov 8 at 21:07






  • 1




    You can extract your database file and open it via sqlitebrowser.org. Then you can run the query and check the execution times.
    – Danail Alexiev
    Nov 8 at 22:01










  • I extracted the DB file and the query using the debugger. And on my computer the query runs in 0ms, returns 3 rows. I should note the table actually has half a million rows.
    – Charon
    Nov 9 at 1:12












  • Is there a difference in the statements / parameters that cause the speed to drop or they are just the same as the quick queries?
    – Danail Alexiev
    Nov 9 at 10:19














  • 1




    Have you tried running the same query in a database tool? Does it run in constant time there?
    – Danail Alexiev
    Nov 8 at 21:05










  • Is there a recommended tool for testing in such a manner on a phone? Ill have to extract the query from the querybuilder I am using.
    – Charon
    Nov 8 at 21:07






  • 1




    You can extract your database file and open it via sqlitebrowser.org. Then you can run the query and check the execution times.
    – Danail Alexiev
    Nov 8 at 22:01










  • I extracted the DB file and the query using the debugger. And on my computer the query runs in 0ms, returns 3 rows. I should note the table actually has half a million rows.
    – Charon
    Nov 9 at 1:12












  • Is there a difference in the statements / parameters that cause the speed to drop or they are just the same as the quick queries?
    – Danail Alexiev
    Nov 9 at 10:19








1




1




Have you tried running the same query in a database tool? Does it run in constant time there?
– Danail Alexiev
Nov 8 at 21:05




Have you tried running the same query in a database tool? Does it run in constant time there?
– Danail Alexiev
Nov 8 at 21:05












Is there a recommended tool for testing in such a manner on a phone? Ill have to extract the query from the querybuilder I am using.
– Charon
Nov 8 at 21:07




Is there a recommended tool for testing in such a manner on a phone? Ill have to extract the query from the querybuilder I am using.
– Charon
Nov 8 at 21:07




1




1




You can extract your database file and open it via sqlitebrowser.org. Then you can run the query and check the execution times.
– Danail Alexiev
Nov 8 at 22:01




You can extract your database file and open it via sqlitebrowser.org. Then you can run the query and check the execution times.
– Danail Alexiev
Nov 8 at 22:01












I extracted the DB file and the query using the debugger. And on my computer the query runs in 0ms, returns 3 rows. I should note the table actually has half a million rows.
– Charon
Nov 9 at 1:12






I extracted the DB file and the query using the debugger. And on my computer the query runs in 0ms, returns 3 rows. I should note the table actually has half a million rows.
– Charon
Nov 9 at 1:12














Is there a difference in the statements / parameters that cause the speed to drop or they are just the same as the quick queries?
– Danail Alexiev
Nov 9 at 10:19




Is there a difference in the statements / parameters that cause the speed to drop or they are just the same as the quick queries?
– Danail Alexiev
Nov 9 at 10:19

















active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














 

draft saved


draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53215952%2finconsistent-query-execution-time%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown






























active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes
















 

draft saved


draft discarded



















































 


draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53215952%2finconsistent-query-execution-time%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

How to pass form data using jquery Ajax to insert data in database?

National Museum of Racing and Hall of Fame

Guess what letter conforming each word