Grafana influxdb error bad gateway



[Bug] Constant 502 Bad Gateway from Grafana After 4.0 Upgrade #6790

Comments

willneumob commented Dec 2, 2016 •

What Grafana version are you using?

What datasource are you using?

What OS are you running grafana on?

Ubuntu 16.04, behind nginx reverse proxy on same system.

Load a dashboard and leave it open for a few minutes.

Not have every panel error on refresh.

  • What happened instead?
    After I leave any dashboard open for more than a few minutes, all panels get errors. Once I refresh the page, the panels load. Did not have any such problems before 4.0 upgrade.

An image or text representation of your metric query

aliasByNode(summarize(removeBelowValue(highestCurrent(sys.*.statsd.gauge-mygauge.total_something, 10), 1), ‘$Interval’, ‘avg’), 1)

The raw query and response for the network request

[pid 6684] write(139, «HTTP/1.0 502 Bad Gateway\r\nDate: Fri, 02 Dec 2016 00:28:56 GMT\r\nContent-Length: 0\r\nContent-Type: text/plain; charset=utf-8\r\n\r\n», 125

Grafana itself is returning 502 (not nginx), and even with logging set to Debug for file , there is no meaningful data or errors reported in /var/log/grafana/grafana.log . I realize there’s a lack of good debugging information here, but without logs, I’m not really sure how to narrow it down.

I’m okay with downgrading, but can’t find any documentation with regard to schema changes and if a downgrade would cause things to break. I can provide more debugging information if pointed in the right direction. Thanks!

The text was updated successfully, but these errors were encountered:

Источник

Bad gate way in Grafana when connecting to InfluxDB #289

Comments

ducminhnguyen commented Jan 22, 2019

Please run `microk8s.inspect
inspection-report-20190122_192827.tar.gz
We appreciate your feedback. Thank you for using microk8s.

I cannot get the influxdb to grafana. It always return 502 bad gateway error

The text was updated successfully, but these errors were encountered:

ktsakalozos commented Jan 22, 2019

Can you please describe what you are deploying and how it fails? Can you share the manifests you are applying and/or instructions on how your setup is supposed to look like and how it is failing? I basically asking for a way to reproduce the issue you are seeing.

heckad commented Nov 27, 2019

Hello, everybody. I have the same error. I only eval microk8s.enable prometheus and add the port forward to svc/grafana 3000 .

ktsakalozos commented Nov 27, 2019

After you enable prometheus and having the pods running, can you access grafana through its ClusterIP (with something like http://10.152.183.153:3000/login ) ?

Can you help me reproduce this? How do you do the port forwarding? Are you running MicroK8s in a VM? Can you share the tarball from microk8s.inspect ? Thanks

heckad commented Nov 27, 2019 •

Yes, I can connect to grafana and success login, but when I try to see metrics I got a red triangle in the corner of the graph. In network tab I found requests with 502 error code on url http://localhost:3000/api/datasources/proxy/1/api/v1/query?query=1%20-%20avg(rate(node_cpu_seconds_total%7Bmode%3D%22idle%22%7D%5B1m%5D))&time=1574896347 .

Port forwarding command: k port-forward -n kubectl monitoring svc/grafana 3000 . Then I forwarded the port to my computer using ssh.

No, I run MicroK8s on my vps.

heckad commented Nov 29, 2019

In grafana logs

ktsakalozos commented Nov 30, 2019

@heckad we updated the prometheus operator addon [1]. Can you give the edge snap a try:

ktsakalozos commented Dec 9, 2019

I assume this is addressed. Please feel free to reopen this issue if you still see problems.

Footer

© 2023 GitHub, Inc.

You can’t perform that action at this time.

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.

Источник

HTTP Error Bad Gateway when using prometheus #14629

Comments

aximo commented Dec 22, 2018

Read before posting:
Please include this information:

What Grafana version are you using?

What datasource are you using?

prometheus latest docke image

What OS are you running grafana on?

Linux 4.4.0-104-generic #127-Ubuntu SMP Mon Dec 11 12:16:42 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

What did you do?

I create a prometheus datasource in grafana web ui

What was the expected result?

success create datasource

What happened instead?

Error with ‘HTTP Error Bad Gateway’

I run grafana and prometheus in docker with almost default configuration. after both started, i check connection in grafana:

i think it means that prometheus endpoint is ready for grafana backend.

then I creete data source in grafana ui like this:

so I think is a bug, or I miss some configuration?

The text was updated successfully, but these errors were encountered:

aximo commented Dec 22, 2018

sorry for my mistake

ZhiruiFeng commented Feb 16, 2019

@aximo Hi, did you find any solution for this problem?

mimarpe commented Feb 25, 2019

I also suffer the same issue, let me know if you found a solution, thanks!

torkelo commented Mar 5, 2019

it’s networking / connection issue. Since it’s not a problem in Grafana we close it.

SamD commented May 8, 2019 •

I have the same problem and if I curl the url from the Grafana back-end (pod) it works perfectly so it isn’t networking related

Читайте также:  Server error in rdweb pages application

I’m new to Grafana & Prometheus my url is the /metrics uri on the service which works fine e.g.

I have done the ingress controller deployment from the helmchart which creates a custom metrics service that exposes 9913 on the ingress controller pod

AndrewWPhillips commented Oct 21, 2019

I had the same problem and found that changing access from Server to Browser fixes it. It appears that the grafana server if being prevented from accessing the address:port while the browser is not blocked.

@torkelo I guess you could argue this is a network issue but you could also consider it a Grafana config issue. I prefer to use the Server option for better security.

souvikhaldar commented Oct 31, 2019

I too was facing it. I’ve solved it but the solution is a bit hilarious. The solution is-
In the URL field, you need to explicitly write the URL (in my case it was http://localhost:9090). Earlier I was not writing it and there already was the url as watermark!

glitch-k8s commented Nov 1, 2019

@souvikhaldar you are absolutely right. Its a water mark and not actual content. Thanks.

jimmycgz commented Dec 6, 2019 •

I still face this error after explicitly writing the URL (in my case it was http://localhost:9090), finally I resolved it by using http://prometheus:9090 or http:/172.17.0.2:9090 because localhost is not regonized as server host when I’m running both Prometheus and Grafana as containers and you should get the correct IP of prometheus container for grafana to connect to, below is the command line I used:

DaCao commented Dec 22, 2019

@jimmycgz Thank you for the answer. It now works.

JoanGil commented Apr 20, 2020 •

Maybe it’s too late to answer but I faced the same problem and I solved it by using: host.docker.internal instead of localhost .

rafael-queiroz-cko commented Jun 5, 2020

just a note, you typed host.internal.docker but it is actually host.docker.internal

JoanGil commented Jun 5, 2020

tunkul commented Sep 12, 2020

I had the same issue being completely new to grafana, just following the tutorial. Changing from «Server» to «Browser» did the trick.

sysadmin-exe commented Oct 2, 2020

Had same issue. First time on grafana. Solved issue by using URL with IP address of Prometheus Docker Pod (docker inspect ) like so url: http://172.17.0.2:9090
This worked instantly.

manero6 commented Oct 2, 2020

I mapped port 8010 to 9090 when running Prometheus (-p 8010:9090).
It finds itself on the default network just as the Grafana container does.

I’m able to connect to it either with:
http://172.17.0.4:9090 (connects to the container directly)
or
http://172.17.0.1:8010 (connects to the mapped port)
or
http://server-ip:8010 (connects to the mapped port)

amitad16 commented Oct 9, 2020

This solution worked for me

First check if port 9090 is working

sudo lsof -i:9090

then check the promethues logs

sudo docker-compose logs promethues

it showed me error global_scrape timeout greater than scrape_interval

they told to increase the scrape_timeout , but nothing about the scrape_interval in /root/bbb-monitoring/prometheus.yaml
So, I increased them to the same value

sudo docker-compose up -d

this should return

then I set the Data Source -> Promethues url to
http://localhost:9090

Hope this helps someone.

neo7337 commented Oct 25, 2020

On running this on local minikube cluster, use the cluster address provided in logs to connect to prometheus server from grafana as the Access is Server(default) and Host clearly states : Your access method is Server, this means the URL needs to be accessible from the grafana backend/server.

lorezi commented Jan 29, 2021

I had the same problem and found that changing access from Server to Browser fixes it. It appears that the grafana server if being prevented from accessing the address:port while the browser is not blocked.

@torkelo I guess you could argue this is a network issue but you could also consider it a Grafana config issue. I prefer to use the Server option for better security.

@AndrewWPhillips changing the access to Browser worked for me. Thanks

xsqian commented Mar 30, 2021

Had same issue. First time on grafana. Solved issue by using URL with IP address of Prometheus Docker Pod (docker inspect ) like so url: http://172.17.0.2:9090
This worked instantly.

also works on your host IP like this HTTP://your_host_ip:9090, note that it’s your host IP, not «localhost», not «127.0.0.1»

RMartires commented Apr 20, 2021 •

use the cluster ip’s so http://172.20.207.29:9090

this is if grafana and prometeus are on kubernetes pods

ahmetonurslmz commented Apr 24, 2021

exact solution for this and all related connection problems between docker images. Connection to prometheus to local app and from grafana to prometheus.
All of these issues is able to solved by using host.internal.docker instead of localhost or local ip

cute-angelia commented Aug 11, 2021

i think it’s problem http Access : select Browser not Server

Читайте также:  What is error 3005

vuhuythanh1995 commented Sep 1, 2021

I had the same issue being completely new to grafana, just following the tutorial. Changing from «Server» to «Browser» did the trick.

I did that but nothing shows up in dashboarh.

Sulmon commented Oct 13, 2021

i had the same issue . In my case i think it happened because i was running graphana and prometheus in the same terminal so one would automatoically shutdown when the other will start. So obviouly graphana for example couldn’t connect with prometheus because it was off. When i open two different terminals and launched each application seperately it worked

stolsvik commented Nov 20, 2021

The tip from @souvikhaldar was spot on. 🤦 . That is just an absurd hurdle for new users of Grafana to have to stumble across!
This github issue is Google’s first hit for «grafana bad gateway prometheus».

Please, @torkelo, if you could just put something other there, like «» or something, or actually let the help text there (http://localhost:9090/) be actual text instead, I guess you would help a lot of first time users like myself. @souvikhaldar’s comment currently has 47 👍 ‘s, and that are only the ones that admit to this being the problem!

stolsvik commented Nov 21, 2021

Wrt. the docker containers — if you run both Prometheus and Grafana with —network=»host» (and then dropping the -p arguments for publishing the ports), it works with http://localhost:9090/ .

rthamrin commented Dec 16, 2021

i think it’s problem http Access : select Browser not Server

this is saving my life 😀

kartikdc commented May 13, 2022

I was facing this problem because I was trying to access localhost:9090 while Grafana was running in a docker container in a bridge network. 🙂

Источник

HTTP Error Bad Gateway when using prometheus #14629

Comments

aximo commented Dec 22, 2018

Read before posting:
Please include this information:

What Grafana version are you using?

What datasource are you using?

prometheus latest docke image

What OS are you running grafana on?

Linux 4.4.0-104-generic #127-Ubuntu SMP Mon Dec 11 12:16:42 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

What did you do?

I create a prometheus datasource in grafana web ui

What was the expected result?

success create datasource

What happened instead?

Error with ‘HTTP Error Bad Gateway’

I run grafana and prometheus in docker with almost default configuration. after both started, i check connection in grafana:

i think it means that prometheus endpoint is ready for grafana backend.

then I creete data source in grafana ui like this:

so I think is a bug, or I miss some configuration?

The text was updated successfully, but these errors were encountered:

aximo commented Dec 22, 2018

sorry for my mistake

ZhiruiFeng commented Feb 16, 2019

@aximo Hi, did you find any solution for this problem?

mimarpe commented Feb 25, 2019

I also suffer the same issue, let me know if you found a solution, thanks!

torkelo commented Mar 5, 2019

it’s networking / connection issue. Since it’s not a problem in Grafana we close it.

SamD commented May 8, 2019 •

I have the same problem and if I curl the url from the Grafana back-end (pod) it works perfectly so it isn’t networking related

I’m new to Grafana & Prometheus my url is the /metrics uri on the service which works fine e.g.

I have done the ingress controller deployment from the helmchart which creates a custom metrics service that exposes 9913 on the ingress controller pod

AndrewWPhillips commented Oct 21, 2019

I had the same problem and found that changing access from Server to Browser fixes it. It appears that the grafana server if being prevented from accessing the address:port while the browser is not blocked.

@torkelo I guess you could argue this is a network issue but you could also consider it a Grafana config issue. I prefer to use the Server option for better security.

souvikhaldar commented Oct 31, 2019

I too was facing it. I’ve solved it but the solution is a bit hilarious. The solution is-
In the URL field, you need to explicitly write the URL (in my case it was http://localhost:9090). Earlier I was not writing it and there already was the url as watermark!

glitch-k8s commented Nov 1, 2019

@souvikhaldar you are absolutely right. Its a water mark and not actual content. Thanks.

jimmycgz commented Dec 6, 2019 •

I still face this error after explicitly writing the URL (in my case it was http://localhost:9090), finally I resolved it by using http://prometheus:9090 or http:/172.17.0.2:9090 because localhost is not regonized as server host when I’m running both Prometheus and Grafana as containers and you should get the correct IP of prometheus container for grafana to connect to, below is the command line I used:

DaCao commented Dec 22, 2019

@jimmycgz Thank you for the answer. It now works.

JoanGil commented Apr 20, 2020 •

Maybe it’s too late to answer but I faced the same problem and I solved it by using: host.docker.internal instead of localhost .

Читайте также:  Out bin full error

rafael-queiroz-cko commented Jun 5, 2020

just a note, you typed host.internal.docker but it is actually host.docker.internal

JoanGil commented Jun 5, 2020

tunkul commented Sep 12, 2020

I had the same issue being completely new to grafana, just following the tutorial. Changing from «Server» to «Browser» did the trick.

sysadmin-exe commented Oct 2, 2020

Had same issue. First time on grafana. Solved issue by using URL with IP address of Prometheus Docker Pod (docker inspect ) like so url: http://172.17.0.2:9090
This worked instantly.

manero6 commented Oct 2, 2020

I mapped port 8010 to 9090 when running Prometheus (-p 8010:9090).
It finds itself on the default network just as the Grafana container does.

I’m able to connect to it either with:
http://172.17.0.4:9090 (connects to the container directly)
or
http://172.17.0.1:8010 (connects to the mapped port)
or
http://server-ip:8010 (connects to the mapped port)

amitad16 commented Oct 9, 2020

This solution worked for me

First check if port 9090 is working

sudo lsof -i:9090

then check the promethues logs

sudo docker-compose logs promethues

it showed me error global_scrape timeout greater than scrape_interval

they told to increase the scrape_timeout , but nothing about the scrape_interval in /root/bbb-monitoring/prometheus.yaml
So, I increased them to the same value

sudo docker-compose up -d

this should return

then I set the Data Source -> Promethues url to
http://localhost:9090

Hope this helps someone.

neo7337 commented Oct 25, 2020

On running this on local minikube cluster, use the cluster address provided in logs to connect to prometheus server from grafana as the Access is Server(default) and Host clearly states : Your access method is Server, this means the URL needs to be accessible from the grafana backend/server.

lorezi commented Jan 29, 2021

I had the same problem and found that changing access from Server to Browser fixes it. It appears that the grafana server if being prevented from accessing the address:port while the browser is not blocked.

@torkelo I guess you could argue this is a network issue but you could also consider it a Grafana config issue. I prefer to use the Server option for better security.

@AndrewWPhillips changing the access to Browser worked for me. Thanks

xsqian commented Mar 30, 2021

Had same issue. First time on grafana. Solved issue by using URL with IP address of Prometheus Docker Pod (docker inspect ) like so url: http://172.17.0.2:9090
This worked instantly.

also works on your host IP like this HTTP://your_host_ip:9090, note that it’s your host IP, not «localhost», not «127.0.0.1»

RMartires commented Apr 20, 2021 •

use the cluster ip’s so http://172.20.207.29:9090

this is if grafana and prometeus are on kubernetes pods

ahmetonurslmz commented Apr 24, 2021

exact solution for this and all related connection problems between docker images. Connection to prometheus to local app and from grafana to prometheus.
All of these issues is able to solved by using host.internal.docker instead of localhost or local ip

cute-angelia commented Aug 11, 2021

i think it’s problem http Access : select Browser not Server

vuhuythanh1995 commented Sep 1, 2021

I had the same issue being completely new to grafana, just following the tutorial. Changing from «Server» to «Browser» did the trick.

I did that but nothing shows up in dashboarh.

Sulmon commented Oct 13, 2021

i had the same issue . In my case i think it happened because i was running graphana and prometheus in the same terminal so one would automatoically shutdown when the other will start. So obviouly graphana for example couldn’t connect with prometheus because it was off. When i open two different terminals and launched each application seperately it worked

stolsvik commented Nov 20, 2021

The tip from @souvikhaldar was spot on. 🤦 . That is just an absurd hurdle for new users of Grafana to have to stumble across!
This github issue is Google’s first hit for «grafana bad gateway prometheus».

Please, @torkelo, if you could just put something other there, like «» or something, or actually let the help text there (http://localhost:9090/) be actual text instead, I guess you would help a lot of first time users like myself. @souvikhaldar’s comment currently has 47 👍 ‘s, and that are only the ones that admit to this being the problem!

stolsvik commented Nov 21, 2021

Wrt. the docker containers — if you run both Prometheus and Grafana with —network=»host» (and then dropping the -p arguments for publishing the ports), it works with http://localhost:9090/ .

rthamrin commented Dec 16, 2021

i think it’s problem http Access : select Browser not Server

this is saving my life 😀

kartikdc commented May 13, 2022

I was facing this problem because I was trying to access localhost:9090 while Grafana was running in a docker container in a bridge network. 🙂

Источник

Оцените статью
toolgir.ru
Adblock
detector