Base docker healthcheck on STDOUT logs
0
I've got a script that outputs logs to STDOUT. I want to monitor this, and if x seconds passes without log output, the healthcheck should fail and the container should restart.
I've got something that I feel should work (using strace), but it's not. I'll post the files below. Does anyone have any idea how to solve this?
Dockerfile:
FROM ruby:2.5.1
RUN apt-get update
RUN dpkg --configure -a
ENV TZ=Europe/London
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y keyboard-configuration
RUN apt-get install -y --fix-missing curl gnupg build-essential mosh iceweasel git libappindicator1 postgresql libpq-dev xvfb xorg dbus-x11
RUN apt-get install -y --fix-missing libxss1 nodejs xfonts-100dpi xfonts-75dpi sqlite libindicator7 unzip wget curl bzip2 libssl-dev libreadline-dev zlib1g-dev strace
RUN ln -sf /usr/bin/nodejs /usr/local/bin/node
RUN gem install bundler activerecord activesupport watir watir-scroll nokogiri pg pry forgery rspec headless rake vcr webmock rails logger
WORKDIR /home/
COPY . .
HEALTHCHECK --interval=25s --timeout=5s --retries=5 CMD strace -p $(pgrep -n ruby) 2>&1 | grep -m 1 write(
CMD ["ruby", "./test.rb"]
test.rb:
require 'logger'
$logger = Logger.new(STDOUT)
$logger.level = Logger::DEBUG
loop do
sleep 5
$logger.warn 'hi'
end
Logs are outputting as expected when running docker container log, but the health status is always starting, and shortly after starting it will consider it failed and restart. I intend to use Docker Swarm once this is working.
ruby docker docker-swarm health-monitoring
asked Nov 20 '18 at 1:43
JoeJoe
120315
add a comment |
0
I've got a script that outputs logs to STDOUT. I want to monitor this, and if x seconds passes without log output, the healthcheck should fail and the container should restart.
I've got something that I feel should work (using strace), but it's not. I'll post the files below. Does anyone have any idea how to solve this?
Dockerfile:
FROM ruby:2.5.1
RUN apt-get update
RUN dpkg --configure -a
ENV TZ=Europe/London
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y keyboard-configuration
RUN apt-get install -y --fix-missing curl gnupg build-essential mosh iceweasel git libappindicator1 postgresql libpq-dev xvfb xorg dbus-x11
RUN apt-get install -y --fix-missing libxss1 nodejs xfonts-100dpi xfonts-75dpi sqlite libindicator7 unzip wget curl bzip2 libssl-dev libreadline-dev zlib1g-dev strace
RUN ln -sf /usr/bin/nodejs /usr/local/bin/node
RUN gem install bundler activerecord activesupport watir watir-scroll nokogiri pg pry forgery rspec headless rake vcr webmock rails logger
WORKDIR /home/
COPY . .
HEALTHCHECK --interval=25s --timeout=5s --retries=5 CMD strace -p $(pgrep -n ruby) 2>&1 | grep -m 1 write(
CMD ["ruby", "./test.rb"]
test.rb:
require 'logger'
$logger = Logger.new(STDOUT)
$logger.level = Logger::DEBUG
loop do
sleep 5
$logger.warn 'hi'
end
Logs are outputting as expected when running docker container log, but the health status is always starting, and shortly after starting it will consider it failed and restart. I intend to use Docker Swarm once this is working.
ruby docker docker-swarm health-monitoring
asked Nov 20 '18 at 1:43
JoeJoe
120315
I'd guess that it's because your loop never finishes, so the script is always running, therefore always in a starting state.
– Rots
Nov 20 '18 at 2:09
@Rots thanks for the comment! Ok. That would explain the starting part, but not the fact that it becomes unhealthy far too early (e.g. with the script above, rather than one that runs for a while and then stops outputting logs before sleeping for a long time, which will also become unhealthy early).
– Joe
Nov 20 '18 at 2:45
No problem. try commenting out the$logger.warnline. Perhaps it's looking for something there and reporting unhealthy. Else you might need to return something like 0 to report success.
– Rots
Nov 20 '18 at 3:46
@Rots unfortunately not, changing the log level to debug still results in it going unhealthy while still outputting logs. I'm less concerned about the 'health: starting' than I am about it becoming unhealthy while logs are still going out.
– Joe
Nov 20 '18 at 10:43
good try. Try setting the exit status withexit 0orexit 1.
– Rots
Nov 20 '18 at 19:56
add a comment |
0
0
0
I've got a script that outputs logs to STDOUT. I want to monitor this, and if x seconds passes without log output, the healthcheck should fail and the container should restart.
I've got something that I feel should work (using strace), but it's not. I'll post the files below. Does anyone have any idea how to solve this?
Dockerfile:
FROM ruby:2.5.1
RUN apt-get update
RUN dpkg --configure -a
ENV TZ=Europe/London
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y keyboard-configuration
RUN apt-get install -y --fix-missing curl gnupg build-essential mosh iceweasel git libappindicator1 postgresql libpq-dev xvfb xorg dbus-x11
RUN apt-get install -y --fix-missing libxss1 nodejs xfonts-100dpi xfonts-75dpi sqlite libindicator7 unzip wget curl bzip2 libssl-dev libreadline-dev zlib1g-dev strace
RUN ln -sf /usr/bin/nodejs /usr/local/bin/node
RUN gem install bundler activerecord activesupport watir watir-scroll nokogiri pg pry forgery rspec headless rake vcr webmock rails logger
WORKDIR /home/
COPY . .
HEALTHCHECK --interval=25s --timeout=5s --retries=5 CMD strace -p $(pgrep -n ruby) 2>&1 | grep -m 1 write(
CMD ["ruby", "./test.rb"]
test.rb:
require 'logger'
$logger = Logger.new(STDOUT)
$logger.level = Logger::DEBUG
loop do
sleep 5
$logger.warn 'hi'
end
Logs are outputting as expected when running docker container log, but the health status is always starting, and shortly after starting it will consider it failed and restart. I intend to use Docker Swarm once this is working.
ruby docker docker-swarm health-monitoring
asked Nov 20 '18 at 1:43
JoeJoe
120315
I've got a script that outputs logs to STDOUT. I want to monitor this, and if x seconds passes without log output, the healthcheck should fail and the container should restart.
I've got something that I feel should work (using strace), but it's not. I'll post the files below. Does anyone have any idea how to solve this?
Dockerfile:
FROM ruby:2.5.1
RUN apt-get update
RUN dpkg --configure -a
ENV TZ=Europe/London
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y keyboard-configuration
RUN apt-get install -y --fix-missing curl gnupg build-essential mosh iceweasel git libappindicator1 postgresql libpq-dev xvfb xorg dbus-x11
RUN apt-get install -y --fix-missing libxss1 nodejs xfonts-100dpi xfonts-75dpi sqlite libindicator7 unzip wget curl bzip2 libssl-dev libreadline-dev zlib1g-dev strace
RUN ln -sf /usr/bin/nodejs /usr/local/bin/node
RUN gem install bundler activerecord activesupport watir watir-scroll nokogiri pg pry forgery rspec headless rake vcr webmock rails logger
WORKDIR /home/
COPY . .
HEALTHCHECK --interval=25s --timeout=5s --retries=5 CMD strace -p $(pgrep -n ruby) 2>&1 | grep -m 1 write(
CMD ["ruby", "./test.rb"]
test.rb:
require 'logger'
$logger = Logger.new(STDOUT)
$logger.level = Logger::DEBUG
loop do
sleep 5
$logger.warn 'hi'
end
Logs are outputting as expected when running docker container log, but the health status is always starting, and shortly after starting it will consider it failed and restart. I intend to use Docker Swarm once this is working.
ruby docker docker-swarm health-monitoring
ruby docker docker-swarm health-monitoring
asked Nov 20 '18 at 1:43
JoeJoe
120315
asked Nov 20 '18 at 1:43
JoeJoe
120315
asked Nov 20 '18 at 1:43
JoeJoe
120315
asked Nov 20 '18 at 1:43
JoeJoe
120315
asked Nov 20 '18 at 1:43
JoeJoe
120315
120315
I'd guess that it's because your loop never finishes, so the script is always running, therefore always in a starting state.
– Rots
Nov 20 '18 at 2:09
@Rots thanks for the comment! Ok. That would explain the starting part, but not the fact that it becomes unhealthy far too early (e.g. with the script above, rather than one that runs for a while and then stops outputting logs before sleeping for a long time, which will also become unhealthy early).
– Joe
Nov 20 '18 at 2:45
No problem. try commenting out the$logger.warnline. Perhaps it's looking for something there and reporting unhealthy. Else you might need to return something like 0 to report success.
– Rots
Nov 20 '18 at 3:46
@Rots unfortunately not, changing the log level to debug still results in it going unhealthy while still outputting logs. I'm less concerned about the 'health: starting' than I am about it becoming unhealthy while logs are still going out.
– Joe
Nov 20 '18 at 10:43
good try. Try setting the exit status withexit 0orexit 1.
– Rots
Nov 20 '18 at 19:56
add a comment |
I'd guess that it's because your loop never finishes, so the script is always running, therefore always in a starting state.
– Rots
Nov 20 '18 at 2:09
@Rots thanks for the comment! Ok. That would explain the starting part, but not the fact that it becomes unhealthy far too early (e.g. with the script above, rather than one that runs for a while and then stops outputting logs before sleeping for a long time, which will also become unhealthy early).
– Joe
Nov 20 '18 at 2:45
No problem. try commenting out the$logger.warnline. Perhaps it's looking for something there and reporting unhealthy. Else you might need to return something like 0 to report success.
– Rots
Nov 20 '18 at 3:46
@Rots unfortunately not, changing the log level to debug still results in it going unhealthy while still outputting logs. I'm less concerned about the 'health: starting' than I am about it becoming unhealthy while logs are still going out.
– Joe
Nov 20 '18 at 10:43
good try. Try setting the exit status withexit 0orexit 1.
– Rots
Nov 20 '18 at 19:56
I'd guess that it's because your loop never finishes, so the script is always running, therefore always in a starting state.
– Rots
Nov 20 '18 at 2:09
I'd guess that it's because your loop never finishes, so the script is always running, therefore always in a starting state.
– Rots
Nov 20 '18 at 2:09
@Rots thanks for the comment! Ok. That would explain the starting part, but not the fact that it becomes unhealthy far too early (e.g. with the script above, rather than one that runs for a while and then stops outputting logs before sleeping for a long time, which will also become unhealthy early).
– Joe
Nov 20 '18 at 2:45
@Rots thanks for the comment! Ok. That would explain the starting part, but not the fact that it becomes unhealthy far too early (e.g. with the script above, rather than one that runs for a while and then stops outputting logs before sleeping for a long time, which will also become unhealthy early).
– Joe
Nov 20 '18 at 2:45
No problem. try commenting out the
$logger.warn line. Perhaps it's looking for something there and reporting unhealthy. Else you might need to return something like 0 to report success.– Rots
Nov 20 '18 at 3:46
No problem. try commenting out the
$logger.warn line. Perhaps it's looking for something there and reporting unhealthy. Else you might need to return something like 0 to report success.– Rots
Nov 20 '18 at 3:46
@Rots unfortunately not, changing the log level to debug still results in it going unhealthy while still outputting logs. I'm less concerned about the 'health: starting' than I am about it becoming unhealthy while logs are still going out.
– Joe
Nov 20 '18 at 10:43
@Rots unfortunately not, changing the log level to debug still results in it going unhealthy while still outputting logs. I'm less concerned about the 'health: starting' than I am about it becoming unhealthy while logs are still going out.
– Joe
Nov 20 '18 at 10:43
good try. Try setting the exit status with
exit 0 or exit 1.– Rots
Nov 20 '18 at 19:56
good try. Try setting the exit status with
exit 0 or exit 1.– Rots
Nov 20 '18 at 19:56
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53385075%2fbase-docker-healthcheck-on-stdout-logs%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53385075%2fbase-docker-healthcheck-on-stdout-logs%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I'd guess that it's because your loop never finishes, so the script is always running, therefore always in a starting state.
– Rots
Nov 20 '18 at 2:09
@Rots thanks for the comment! Ok. That would explain the starting part, but not the fact that it becomes unhealthy far too early (e.g. with the script above, rather than one that runs for a while and then stops outputting logs before sleeping for a long time, which will also become unhealthy early).
– Joe
Nov 20 '18 at 2:45
No problem. try commenting out the
$logger.warnline. Perhaps it's looking for something there and reporting unhealthy. Else you might need to return something like 0 to report success.– Rots
Nov 20 '18 at 3:46
@Rots unfortunately not, changing the log level to debug still results in it going unhealthy while still outputting logs. I'm less concerned about the 'health: starting' than I am about it becoming unhealthy while logs are still going out.
– Joe
Nov 20 '18 at 10:43
good try. Try setting the exit status with
exit 0orexit 1.– Rots
Nov 20 '18 at 19:56