Handling SQL updates for production deployments Announcing the arrival of Valued Associate...
How do I keep my slimes from escaping their pens?
What are the performance impacts of 'functional' Rust?
Jazz greats knew nothing of modes. Why are they used to improvise on standards?
Can I throw a longsword at someone?
Strange behaviour of Check
Windows 10: How to Lock (not sleep) laptop on lid close?
Need a suitable toxic chemical for a murder plot in my novel
How is simplicity better than precision and clarity in prose?
What LEGO pieces have "real-world" functionality?
Using "nakedly" instead of "with nothing on"
Area of a 2D convex hull
How does the Nova's Burn power work at the 7-9 level?
Passing functions in C++
Autumning in love
Why does this iterative way of solving of equation work?
What would be Julian Assange's expected punishment, on the current English criminal law?
How to set letter above or below the symbol?
Statistical model of ligand substitution
What do I do if technical issues prevent me from filing my return on time?
What do you call the holes in a flute?
Why use gamma over alpha radiation?
When communicating altitude with a '9' in it, should it be pronounced "nine hundred" or "niner hundred"?
3 doors, three guards, one stone
Blender game recording at the wrong time
Handling SQL updates for production deployments
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)MySQL Workbench team strategyExport .MWB to working .SQL file using command lineBest practices for deploying and updating SQL Server as part of code deploymentDACPAC and database drift: db users are always detected as drift even when nothing has changedHow to use mysql to migrate database changes from local databases?Builds and deployments to 300+ databases utilizing SSDT/VSTSWhere should I store a custom “.my.cnf” file?SSDT Drop and Recreate Tables when nothing has changedMySQL Schema Change Management Tools?Query is extremely slow after ALTER and deploying of updates
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}
I'm researching for quite a long time now (about two months) and haven't found a good approach for this, so I'll ask you experts out there in the hope of some enlightenment.
I run a pretty traditional LAMP web application in the cloud. The source code is VCS'ed with Git, as well as database schema (split SQL files for structure and default data).
Deploying a new environment is quite a joy, development or production. Dev environments are deployed with Vagrant plus some nice shell provisioning script I wrote. Production environments are deployed using a subset of the Vagrant provisioning script, which is also very easy to do.
When the development cycle imposes a change to the SQL schema (structure or data), things starts to break. In that case, developers changes the core SQL scripts so all the changes gets into the VCS, which is totally fine for brand new environments.
The problem sits within already deployed production environments: for those, I have to manually get all the patches for the SQL changes since the revision actually running in all production servers first so I can apply them manually after. This takes a long time to happen, is a quite fragile task and error prone.
Initially, I thought I can move the SQL changes away from the core SQL files smaller subsets, letting the core SQL files as a starting point. When there is something to change at SQL structure, I could just tell devs to create a new SQL file with only the changes from that development cycle.
So, for instance:
- structure_begin.sql: the SQL schema as it is now.
|- structure_devcycle1.sql: first set of changes to the structure
|- structure_devcycle2.sql: second set of changes to the structure, and so forth...
Then, by using Git hooks, I can selectively and automatically deploy them on production. But I don't know if this is a good approach. And here I can ask:
- Does somebody out there have solved this puzzle?
- What are the best practices for release management and deploy automation for SQL changes in production environments?
- Are there any open source tools that can help in this process? (I'm aware of Sqitch, but I'm not quite sure if it is the right one for the job.)
mysql deployment change-management
bumped to the homepage by Community♦ 8 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
add a comment |
I'm researching for quite a long time now (about two months) and haven't found a good approach for this, so I'll ask you experts out there in the hope of some enlightenment.
I run a pretty traditional LAMP web application in the cloud. The source code is VCS'ed with Git, as well as database schema (split SQL files for structure and default data).
Deploying a new environment is quite a joy, development or production. Dev environments are deployed with Vagrant plus some nice shell provisioning script I wrote. Production environments are deployed using a subset of the Vagrant provisioning script, which is also very easy to do.
When the development cycle imposes a change to the SQL schema (structure or data), things starts to break. In that case, developers changes the core SQL scripts so all the changes gets into the VCS, which is totally fine for brand new environments.
The problem sits within already deployed production environments: for those, I have to manually get all the patches for the SQL changes since the revision actually running in all production servers first so I can apply them manually after. This takes a long time to happen, is a quite fragile task and error prone.
Initially, I thought I can move the SQL changes away from the core SQL files smaller subsets, letting the core SQL files as a starting point. When there is something to change at SQL structure, I could just tell devs to create a new SQL file with only the changes from that development cycle.
So, for instance:
- structure_begin.sql: the SQL schema as it is now.
|- structure_devcycle1.sql: first set of changes to the structure
|- structure_devcycle2.sql: second set of changes to the structure, and so forth...
Then, by using Git hooks, I can selectively and automatically deploy them on production. But I don't know if this is a good approach. And here I can ask:
- Does somebody out there have solved this puzzle?
- What are the best practices for release management and deploy automation for SQL changes in production environments?
- Are there any open source tools that can help in this process? (I'm aware of Sqitch, but I'm not quite sure if it is the right one for the job.)
mysql deployment change-management
bumped to the homepage by Community♦ 8 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
PostgreSQL has a VERY cool feature for this. we can do transactional DDLs. this means that you can rollback any create table, create index, alter table, etc. this helps to avoid problems related to syntax errors and all that. but, this is not all. CREATE EXTENSION (postgresql.org/docs/9.4/static/sql-createextension.html). it allows you to version stuff. so you can directly jump from version to the other. no need to say that it is transactional as well. consider switching - mysql cannot do this kind of stuff.
– Hans-Jürgen Schönig
Nov 15 '14 at 9:29
Indeed, this is a very nice feature of PostgreSQL, @Hans-JürgenSchönig. Unfortunately, the application I manage is quite an old one, do not use a DB HAL, so just migrate it is quite a huge job for now and is not in the sight for at least the medium term (1~2y). I'm looking for a solution that could be implemented with the present stack, where MySQL/MariaDB is the main RDBMS.
– Deny Dias
Nov 16 '14 at 17:54
I would suggest reading this article on InfoQ it touches on a number of points brought up in this discussion
– user3660241
Nov 23 '14 at 16:10
That article is gold, @user3660241! Thanks for sharing it!
– Deny Dias
Nov 24 '14 at 20:08
add a comment |
I'm researching for quite a long time now (about two months) and haven't found a good approach for this, so I'll ask you experts out there in the hope of some enlightenment.
I run a pretty traditional LAMP web application in the cloud. The source code is VCS'ed with Git, as well as database schema (split SQL files for structure and default data).
Deploying a new environment is quite a joy, development or production. Dev environments are deployed with Vagrant plus some nice shell provisioning script I wrote. Production environments are deployed using a subset of the Vagrant provisioning script, which is also very easy to do.
When the development cycle imposes a change to the SQL schema (structure or data), things starts to break. In that case, developers changes the core SQL scripts so all the changes gets into the VCS, which is totally fine for brand new environments.
The problem sits within already deployed production environments: for those, I have to manually get all the patches for the SQL changes since the revision actually running in all production servers first so I can apply them manually after. This takes a long time to happen, is a quite fragile task and error prone.
Initially, I thought I can move the SQL changes away from the core SQL files smaller subsets, letting the core SQL files as a starting point. When there is something to change at SQL structure, I could just tell devs to create a new SQL file with only the changes from that development cycle.
So, for instance:
- structure_begin.sql: the SQL schema as it is now.
|- structure_devcycle1.sql: first set of changes to the structure
|- structure_devcycle2.sql: second set of changes to the structure, and so forth...
Then, by using Git hooks, I can selectively and automatically deploy them on production. But I don't know if this is a good approach. And here I can ask:
- Does somebody out there have solved this puzzle?
- What are the best practices for release management and deploy automation for SQL changes in production environments?
- Are there any open source tools that can help in this process? (I'm aware of Sqitch, but I'm not quite sure if it is the right one for the job.)
mysql deployment change-management
I'm researching for quite a long time now (about two months) and haven't found a good approach for this, so I'll ask you experts out there in the hope of some enlightenment.
I run a pretty traditional LAMP web application in the cloud. The source code is VCS'ed with Git, as well as database schema (split SQL files for structure and default data).
Deploying a new environment is quite a joy, development or production. Dev environments are deployed with Vagrant plus some nice shell provisioning script I wrote. Production environments are deployed using a subset of the Vagrant provisioning script, which is also very easy to do.
When the development cycle imposes a change to the SQL schema (structure or data), things starts to break. In that case, developers changes the core SQL scripts so all the changes gets into the VCS, which is totally fine for brand new environments.
The problem sits within already deployed production environments: for those, I have to manually get all the patches for the SQL changes since the revision actually running in all production servers first so I can apply them manually after. This takes a long time to happen, is a quite fragile task and error prone.
Initially, I thought I can move the SQL changes away from the core SQL files smaller subsets, letting the core SQL files as a starting point. When there is something to change at SQL structure, I could just tell devs to create a new SQL file with only the changes from that development cycle.
So, for instance:
- structure_begin.sql: the SQL schema as it is now.
|- structure_devcycle1.sql: first set of changes to the structure
|- structure_devcycle2.sql: second set of changes to the structure, and so forth...
Then, by using Git hooks, I can selectively and automatically deploy them on production. But I don't know if this is a good approach. And here I can ask:
- Does somebody out there have solved this puzzle?
- What are the best practices for release management and deploy automation for SQL changes in production environments?
- Are there any open source tools that can help in this process? (I'm aware of Sqitch, but I'm not quite sure if it is the right one for the job.)
mysql deployment change-management
mysql deployment change-management
edited Mar 24 '16 at 19:35
Deny Dias
asked Nov 14 '14 at 23:29
Deny DiasDeny Dias
213
213
bumped to the homepage by Community♦ 8 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
bumped to the homepage by Community♦ 8 mins ago
This question has answers that may be good or bad; the system has marked it active so that they can be reviewed.
PostgreSQL has a VERY cool feature for this. we can do transactional DDLs. this means that you can rollback any create table, create index, alter table, etc. this helps to avoid problems related to syntax errors and all that. but, this is not all. CREATE EXTENSION (postgresql.org/docs/9.4/static/sql-createextension.html). it allows you to version stuff. so you can directly jump from version to the other. no need to say that it is transactional as well. consider switching - mysql cannot do this kind of stuff.
– Hans-Jürgen Schönig
Nov 15 '14 at 9:29
Indeed, this is a very nice feature of PostgreSQL, @Hans-JürgenSchönig. Unfortunately, the application I manage is quite an old one, do not use a DB HAL, so just migrate it is quite a huge job for now and is not in the sight for at least the medium term (1~2y). I'm looking for a solution that could be implemented with the present stack, where MySQL/MariaDB is the main RDBMS.
– Deny Dias
Nov 16 '14 at 17:54
I would suggest reading this article on InfoQ it touches on a number of points brought up in this discussion
– user3660241
Nov 23 '14 at 16:10
That article is gold, @user3660241! Thanks for sharing it!
– Deny Dias
Nov 24 '14 at 20:08
add a comment |
PostgreSQL has a VERY cool feature for this. we can do transactional DDLs. this means that you can rollback any create table, create index, alter table, etc. this helps to avoid problems related to syntax errors and all that. but, this is not all. CREATE EXTENSION (postgresql.org/docs/9.4/static/sql-createextension.html). it allows you to version stuff. so you can directly jump from version to the other. no need to say that it is transactional as well. consider switching - mysql cannot do this kind of stuff.
– Hans-Jürgen Schönig
Nov 15 '14 at 9:29
Indeed, this is a very nice feature of PostgreSQL, @Hans-JürgenSchönig. Unfortunately, the application I manage is quite an old one, do not use a DB HAL, so just migrate it is quite a huge job for now and is not in the sight for at least the medium term (1~2y). I'm looking for a solution that could be implemented with the present stack, where MySQL/MariaDB is the main RDBMS.
– Deny Dias
Nov 16 '14 at 17:54
I would suggest reading this article on InfoQ it touches on a number of points brought up in this discussion
– user3660241
Nov 23 '14 at 16:10
That article is gold, @user3660241! Thanks for sharing it!
– Deny Dias
Nov 24 '14 at 20:08
PostgreSQL has a VERY cool feature for this. we can do transactional DDLs. this means that you can rollback any create table, create index, alter table, etc. this helps to avoid problems related to syntax errors and all that. but, this is not all. CREATE EXTENSION (postgresql.org/docs/9.4/static/sql-createextension.html). it allows you to version stuff. so you can directly jump from version to the other. no need to say that it is transactional as well. consider switching - mysql cannot do this kind of stuff.
– Hans-Jürgen Schönig
Nov 15 '14 at 9:29
PostgreSQL has a VERY cool feature for this. we can do transactional DDLs. this means that you can rollback any create table, create index, alter table, etc. this helps to avoid problems related to syntax errors and all that. but, this is not all. CREATE EXTENSION (postgresql.org/docs/9.4/static/sql-createextension.html). it allows you to version stuff. so you can directly jump from version to the other. no need to say that it is transactional as well. consider switching - mysql cannot do this kind of stuff.
– Hans-Jürgen Schönig
Nov 15 '14 at 9:29
Indeed, this is a very nice feature of PostgreSQL, @Hans-JürgenSchönig. Unfortunately, the application I manage is quite an old one, do not use a DB HAL, so just migrate it is quite a huge job for now and is not in the sight for at least the medium term (1~2y). I'm looking for a solution that could be implemented with the present stack, where MySQL/MariaDB is the main RDBMS.
– Deny Dias
Nov 16 '14 at 17:54
Indeed, this is a very nice feature of PostgreSQL, @Hans-JürgenSchönig. Unfortunately, the application I manage is quite an old one, do not use a DB HAL, so just migrate it is quite a huge job for now and is not in the sight for at least the medium term (1~2y). I'm looking for a solution that could be implemented with the present stack, where MySQL/MariaDB is the main RDBMS.
– Deny Dias
Nov 16 '14 at 17:54
I would suggest reading this article on InfoQ it touches on a number of points brought up in this discussion
– user3660241
Nov 23 '14 at 16:10
I would suggest reading this article on InfoQ it touches on a number of points brought up in this discussion
– user3660241
Nov 23 '14 at 16:10
That article is gold, @user3660241! Thanks for sharing it!
– Deny Dias
Nov 24 '14 at 20:08
That article is gold, @user3660241! Thanks for sharing it!
– Deny Dias
Nov 24 '14 at 20:08
add a comment |
1 Answer
1
active
oldest
votes
I am not sure if you still need an answer, but I had the same problem with managing and deploying schema changes. Sqitch was the first thing I tried to mechanize migrations, but I did not really like it.
Just as you described, we have separate .sql files with CREATE TABLE and other instructions reflecting our database schema. We created simple tool to generate SQL patches when schema changes. Take a look at it: https://github.com/condograde/sqlibrist. There is detailed tutorial, how to use it.
Hi, Serj! First, sqlibrist is a hell of a name! Congratulations to the guy/gal that came with this up! Yes, the original situation is now under control. But sqlibrist could definitively improve it. I'm not going to hit the accepted answer button now because I have to investigate it further, which will take a while. But for sure your answer deserves my upvote. ;)
– Deny Dias
Mar 24 '16 at 19:31
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "182"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f82704%2fhandling-sql-updates-for-production-deployments%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I am not sure if you still need an answer, but I had the same problem with managing and deploying schema changes. Sqitch was the first thing I tried to mechanize migrations, but I did not really like it.
Just as you described, we have separate .sql files with CREATE TABLE and other instructions reflecting our database schema. We created simple tool to generate SQL patches when schema changes. Take a look at it: https://github.com/condograde/sqlibrist. There is detailed tutorial, how to use it.
Hi, Serj! First, sqlibrist is a hell of a name! Congratulations to the guy/gal that came with this up! Yes, the original situation is now under control. But sqlibrist could definitively improve it. I'm not going to hit the accepted answer button now because I have to investigate it further, which will take a while. But for sure your answer deserves my upvote. ;)
– Deny Dias
Mar 24 '16 at 19:31
add a comment |
I am not sure if you still need an answer, but I had the same problem with managing and deploying schema changes. Sqitch was the first thing I tried to mechanize migrations, but I did not really like it.
Just as you described, we have separate .sql files with CREATE TABLE and other instructions reflecting our database schema. We created simple tool to generate SQL patches when schema changes. Take a look at it: https://github.com/condograde/sqlibrist. There is detailed tutorial, how to use it.
Hi, Serj! First, sqlibrist is a hell of a name! Congratulations to the guy/gal that came with this up! Yes, the original situation is now under control. But sqlibrist could definitively improve it. I'm not going to hit the accepted answer button now because I have to investigate it further, which will take a while. But for sure your answer deserves my upvote. ;)
– Deny Dias
Mar 24 '16 at 19:31
add a comment |
I am not sure if you still need an answer, but I had the same problem with managing and deploying schema changes. Sqitch was the first thing I tried to mechanize migrations, but I did not really like it.
Just as you described, we have separate .sql files with CREATE TABLE and other instructions reflecting our database schema. We created simple tool to generate SQL patches when schema changes. Take a look at it: https://github.com/condograde/sqlibrist. There is detailed tutorial, how to use it.
I am not sure if you still need an answer, but I had the same problem with managing and deploying schema changes. Sqitch was the first thing I tried to mechanize migrations, but I did not really like it.
Just as you described, we have separate .sql files with CREATE TABLE and other instructions reflecting our database schema. We created simple tool to generate SQL patches when schema changes. Take a look at it: https://github.com/condograde/sqlibrist. There is detailed tutorial, how to use it.
answered Mar 23 '16 at 13:36
Serhii ZavadskyiSerhii Zavadskyi
1
1
Hi, Serj! First, sqlibrist is a hell of a name! Congratulations to the guy/gal that came with this up! Yes, the original situation is now under control. But sqlibrist could definitively improve it. I'm not going to hit the accepted answer button now because I have to investigate it further, which will take a while. But for sure your answer deserves my upvote. ;)
– Deny Dias
Mar 24 '16 at 19:31
add a comment |
Hi, Serj! First, sqlibrist is a hell of a name! Congratulations to the guy/gal that came with this up! Yes, the original situation is now under control. But sqlibrist could definitively improve it. I'm not going to hit the accepted answer button now because I have to investigate it further, which will take a while. But for sure your answer deserves my upvote. ;)
– Deny Dias
Mar 24 '16 at 19:31
Hi, Serj! First, sqlibrist is a hell of a name! Congratulations to the guy/gal that came with this up! Yes, the original situation is now under control. But sqlibrist could definitively improve it. I'm not going to hit the accepted answer button now because I have to investigate it further, which will take a while. But for sure your answer deserves my upvote. ;)
– Deny Dias
Mar 24 '16 at 19:31
Hi, Serj! First, sqlibrist is a hell of a name! Congratulations to the guy/gal that came with this up! Yes, the original situation is now under control. But sqlibrist could definitively improve it. I'm not going to hit the accepted answer button now because I have to investigate it further, which will take a while. But for sure your answer deserves my upvote. ;)
– Deny Dias
Mar 24 '16 at 19:31
add a comment |
Thanks for contributing an answer to Database Administrators Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fdba.stackexchange.com%2fquestions%2f82704%2fhandling-sql-updates-for-production-deployments%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
PostgreSQL has a VERY cool feature for this. we can do transactional DDLs. this means that you can rollback any create table, create index, alter table, etc. this helps to avoid problems related to syntax errors and all that. but, this is not all. CREATE EXTENSION (postgresql.org/docs/9.4/static/sql-createextension.html). it allows you to version stuff. so you can directly jump from version to the other. no need to say that it is transactional as well. consider switching - mysql cannot do this kind of stuff.
– Hans-Jürgen Schönig
Nov 15 '14 at 9:29
Indeed, this is a very nice feature of PostgreSQL, @Hans-JürgenSchönig. Unfortunately, the application I manage is quite an old one, do not use a DB HAL, so just migrate it is quite a huge job for now and is not in the sight for at least the medium term (1~2y). I'm looking for a solution that could be implemented with the present stack, where MySQL/MariaDB is the main RDBMS.
– Deny Dias
Nov 16 '14 at 17:54
I would suggest reading this article on InfoQ it touches on a number of points brought up in this discussion
– user3660241
Nov 23 '14 at 16:10
That article is gold, @user3660241! Thanks for sharing it!
– Deny Dias
Nov 24 '14 at 20:08