Optimizing for Modulo QueriesCan MySQL reasonably perform queries on billions of rows?MySQL Read Speed and...
What is the purpose of easy combat scenarios that don't need resource expenditure?
Planet at the end of Solo: A Star Wars Story
Why did the villain in the first Men in Black movie care about Earth's Cockroaches?
I am on the US no-fly list. What can I do in order to be allowed on flights which go through US airspace?
Eww, those bytes are gross
How do creatures spend Hit Dice after a short rest (if they can do so)?
Dilemma of explaining to interviewer that he is the reason for declining second interview
How did the original light saber work?
How to define new environment, using proof-environment
Why can a 352GB NumPy ndarray be used on an 8GB memory macOS computer?
How can I improve my fireworks photography?
Why did Jodrell Bank assist the Soviet Union to collect data from their spacecraft in the mid 1960's?
Does my logo design convey the right feelings for a University Student's Council?
Why do members of Congress in committee hearings ask witnesses the same question multiple times?
En Passant For Beginners
Does Windows 10's telemetry include sending *.doc files if Word crashed?
Knowing when to use pictures over words
Is there a better way to make this?
Why zero tolerance on nudity in space?
What makes the Forgotten Realms "forgotten"?
How can animals be objects of ethics without being subjects as well?
Can a hotel cancel a confirmed reservation?
page split between longtable caption and table
How to generate a matrix with certain conditions
Optimizing for Modulo Queries
Can MySQL reasonably perform queries on billions of rows?MySQL Read Speed and Partitioning on Separate DrivesOptimizing multiple MySQL sub-queries and joins for multiple runsMySQL join query indexesHigh volume data aggregationOptimizing SELECT queriesoptimizing query for slow queriesChoosing index fillfactor for large, highly inserted tableRegularly updating table with huge amount of rowsWhich data model/schema to apply to a timeseries datastore for data sources with varying fields
I have a use case where I want to be able to query tens of thousands of records for the modulo of a certain field value with another number, i.e. queries like
SELECT * FROM EVENTS WHERE MOD(OFFSET)=5
What are some performance considerations, if any, I should have in place to ensure that my queries are more efficient? Are there any specific kinds of indexes that would speed up this kind of querying?
Has there been a published comparative analysis or benchmark or some kind that has been done for these kinds of queries between various databases? Alternatively, is there a naive way of determining what choice of database would be best suited for this kind of querying?
mysql postgresql
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
bumped to the homepage by Community♦ 7 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
I have a use case where I want to be able to query tens of thousands of records for the modulo of a certain field value with another number, i.e. queries like
SELECT * FROM EVENTS WHERE MOD(OFFSET)=5
What are some performance considerations, if any, I should have in place to ensure that my queries are more efficient? Are there any specific kinds of indexes that would speed up this kind of querying?
Has there been a published comparative analysis or benchmark or some kind that has been done for these kinds of queries between various databases? Alternatively, is there a naive way of determining what choice of database would be best suited for this kind of querying?
mysql postgresql
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
bumped to the homepage by Community♦ 7 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
I have a use case where I want to be able to query tens of thousands of records for the modulo of a certain field value with another number, i.e. queries like
SELECT * FROM EVENTS WHERE MOD(OFFSET)=5
What are some performance considerations, if any, I should have in place to ensure that my queries are more efficient? Are there any specific kinds of indexes that would speed up this kind of querying?
Has there been a published comparative analysis or benchmark or some kind that has been done for these kinds of queries between various databases? Alternatively, is there a naive way of determining what choice of database would be best suited for this kind of querying?
mysql postgresql
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
I have a use case where I want to be able to query tens of thousands of records for the modulo of a certain field value with another number, i.e. queries like
SELECT * FROM EVENTS WHERE MOD(OFFSET)=5
What are some performance considerations, if any, I should have in place to ensure that my queries are more efficient? Are there any specific kinds of indexes that would speed up this kind of querying?
Has there been a published comparative analysis or benchmark or some kind that has been done for these kinds of queries between various databases? Alternatively, is there a naive way of determining what choice of database would be best suited for this kind of querying?
mysql postgresql
mysql postgresql
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
asked Dec 10 '14 at 20:09
Fareesh VijayarangamFareesh Vijayarangam
1011
1011
bumped to the homepage by Community♦ 7 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ 7 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
In Postgres, you can use an index on the expression:
CREATE INDEX foo_idx ON events (mod(offset, 5));
Only works for that number, of course. And 5 would be a bad example, because ~ 20 % of the rows would qualify for each possible value, and an index starts paying off for ~ 5 % selected rows or fewer. Otherwise, a sequential scan will be faster.
Aside: offset is a reserved word. Don't use it as identifier.
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "182"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f84841%2foptimizing-for-modulo-queries%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
In Postgres, you can use an index on the expression:
CREATE INDEX foo_idx ON events (mod(offset, 5));
Only works for that number, of course. And 5 would be a bad example, because ~ 20 % of the rows would qualify for each possible value, and an index starts paying off for ~ 5 % selected rows or fewer. Otherwise, a sequential scan will be faster.
Aside: offset is a reserved word. Don't use it as identifier.
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
add a comment |
In Postgres, you can use an index on the expression:
CREATE INDEX foo_idx ON events (mod(offset, 5));
Only works for that number, of course. And 5 would be a bad example, because ~ 20 % of the rows would qualify for each possible value, and an index starts paying off for ~ 5 % selected rows or fewer. Otherwise, a sequential scan will be faster.
Aside: offset is a reserved word. Don't use it as identifier.
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
add a comment |
In Postgres, you can use an index on the expression:
CREATE INDEX foo_idx ON events (mod(offset, 5));
Only works for that number, of course. And 5 would be a bad example, because ~ 20 % of the rows would qualify for each possible value, and an index starts paying off for ~ 5 % selected rows or fewer. Otherwise, a sequential scan will be faster.
Aside: offset is a reserved word. Don't use it as identifier.
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
In Postgres, you can use an index on the expression:
CREATE INDEX foo_idx ON events (mod(offset, 5));
Only works for that number, of course. And 5 would be a bad example, because ~ 20 % of the rows would qualify for each possible value, and an index starts paying off for ~ 5 % selected rows or fewer. Otherwise, a sequential scan will be faster.
Aside: offset is a reserved word. Don't use it as identifier.
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
edited Dec 10 '14 at 20:40
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
answered Dec 10 '14 at 20:34
Erwin BrandstetterErwin Brandstetter
93.6k9180294
93.6k9180294
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
add a comment |
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
The number would not be constant unfortunately
– Fareesh Vijayarangam
Dec 10 '14 at 23:39
add a comment |
Thanks for contributing an answer to Database Administrators Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f84841%2foptimizing-for-modulo-queries%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
