Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
G
GitlabRSSSync
Manage
Activity
Members
Labels
Plan
Issues
0
Issue boards
Milestones
Iterations
Requirements
Code
Merge requests
0
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Locked files
Build
Pipelines
Jobs
Pipeline schedules
Test cases
Artifacts
Deploy
Releases
Container Registry
Model registry
Operate
Environments
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Code review analytics
Issue analytics
Insights
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
open-source
GitlabRSSSync
Commits
bd9fec2c
Commit
bd9fec2c
authored
5 years ago
by
Adam Harrison-Fuller
Browse files
Options
Downloads
Patches
Plain Diff
Adding Sentinel support and a /healthz endpoint
parent
ef2d9bb9
No related branches found
Branches containing commit
No related tags found
Tags containing commit
No related merge requests found
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
README.md
+5
-1
5 additions, 1 deletion
README.md
main.go
+42
-7
42 additions, 7 deletions
main.go
with
47 additions
and
8 deletions
README.md
+
5
−
1
View file @
bd9fec2c
...
...
@@ -63,12 +63,13 @@ issues in the projects you specify in the config file.
*
CONFIG_DIR - The directory the application should look for config.yaml in.
*
REDIS_URL - The URL of the Redis host e.g.
`redis:6379`
*
REDIS_PASSWORD - Password for Redis, if an empty password is required set to
`REDIS_PASSWORD=`
*
USE_SENTINEL - If set the REDIS_URL will be treated as a sentinel and the current master acquired via the sentinel.
### Run it
#### Via Docker
```
bash
docker run
-e
GITLAB_API_TOKEN
=
<INSERT_TOKEN>
-e
CONFIG_DIR
=
/app
-v
REDIS_URL
=
<REDIS_URL>
-v
REDIS_PASSWORD
=
<REDIS_PASSWORD>
-v
<PATH_TO_CONFIG_DIR>
/config adamhf/rss-sync:latest
docker run
-e
GITLAB_API_TOKEN
=
<INSERT_TOKEN>
-e
CONFIG_DIR
=
/app
-v
REDIS_URL
=
<REDIS_URL>
-v
REDIS_PASSWORD
=
<REDIS_PASSWORD>
-v
${
PWD
}
:
/config adamhf/rss-sync:latest
```
#### Via docker-compose
...
...
@@ -81,6 +82,9 @@ Two metrics (above and beyond what are exposed by the Go Prometheus library) are
*
last_run_time - The time of the last feed checks, useful for creating alerts to check for successful runs.
*
issues_created - The total number of issues created in Gitlab, useful to check for runaways.
## Healthz Endpoint
A /healthz endpoint is exposed on :8081/healthz which will fail if it is unable to connect to Redis.
## Example Issues
### GKE Release Notes
Feed URL: https://cloud.google.com/feeds/kubernetes-engine-release-notes.xml
...
...
This diff is collapsed.
Click to expand it.
main.go
+
42
−
7
View file @
bd9fec2c
...
...
@@ -42,6 +42,7 @@ type EnvValues struct {
RedisPassword
string
ConfDir
string
GitlabAPIKey
string
UseSentinel
bool
}
func
hasExistingGitlabIssue
(
guid
string
,
projectID
int
,
gitlabClient
*
gitlab
.
Client
)
bool
{
...
...
@@ -84,7 +85,7 @@ func (feed Feed) checkFeed(redisClient *redis.Client, gitlabClient *gitlab.Clien
var
oldArticle
[]
*
gofeed
.
Item
for
_
,
item
:=
range
rss
.
Items
{
found
:=
redisClient
.
SIsMember
(
feed
.
ID
,
item
.
GUID
)
.
Val
()
if
found
==
true
{
if
found
{
oldArticle
=
append
(
oldArticle
,
item
)
}
else
{
newArticle
=
append
(
newArticle
,
item
)
...
...
@@ -187,11 +188,20 @@ func initialise(env EnvValues) (redisClient *redis.Client, client *gitlab.Client
client
=
gitlab
.
NewClient
(
nil
,
env
.
GitlabAPIKey
)
config
=
readConfig
(
path
.
Join
(
env
.
ConfDir
,
"config.yaml"
))
redisClient
=
redis
.
NewClient
(
&
redis
.
Options
{
Addr
:
env
.
RedisURL
,
Password
:
env
.
RedisPassword
,
DB
:
0
,
// use default DB
})
if
!
env
.
UseSentinel
{
redisClient
=
redis
.
NewClient
(
&
redis
.
Options
{
Addr
:
env
.
RedisURL
,
Password
:
env
.
RedisPassword
,
DB
:
0
,
// use default DB
})
}
else
{
redisClient
=
redis
.
NewFailoverClient
(
&
redis
.
FailoverOptions
{
SentinelAddrs
:
[]
string
{
env
.
RedisURL
},
Password
:
env
.
RedisPassword
,
MasterName
:
"mymaster"
,
DB
:
0
,
// use default DB
})
}
if
err
:=
redisClient
.
Ping
()
.
Err
();
err
!=
nil
{
panic
(
fmt
.
Sprintf
(
"Unable to connect to Redis @ %s"
,
env
.
RedisURL
))
...
...
@@ -205,7 +215,7 @@ func initialise(env EnvValues) (redisClient *redis.Client, client *gitlab.Client
func
main
()
{
env
:=
readEnv
()
redisClient
,
gitlabClient
,
config
:=
initialise
(
env
)
go
checkLiveliness
(
redisClient
)
go
func
()
{
for
{
log
.
Printf
(
"Running checks at %s
\n
"
,
time
.
Now
()
.
Format
(
time
.
RFC850
))
...
...
@@ -224,6 +234,8 @@ func main() {
func
readEnv
()
EnvValues
{
var
gitlabPAToken
,
configDir
,
redisURL
,
redisPassword
string
useSentinel
:=
false
if
envGitlabAPIToken
:=
os
.
Getenv
(
"GITLAB_API_TOKEN"
);
envGitlabAPIToken
==
""
{
panic
(
"Could not find GITLAB_API_TOKEN specified as an environment variable"
)
}
else
{
...
...
@@ -247,10 +259,33 @@ func readEnv() EnvValues {
redisPassword
=
envRedisPassword
}
_
,
hasRedisSentinel
:=
os
.
LookupEnv
(
"USE_SENTINEL"
)
if
hasRedisSentinel
{
log
.
Printf
(
"Running in sentinel mode"
)
useSentinel
=
true
}
return
EnvValues
{
RedisURL
:
redisURL
,
RedisPassword
:
redisPassword
,
ConfDir
:
configDir
,
GitlabAPIKey
:
gitlabPAToken
,
UseSentinel
:
useSentinel
,
}
}
func
checkLiveliness
(
client
*
redis
.
Client
)
{
http
.
HandleFunc
(
"/healthz"
,
func
(
w
http
.
ResponseWriter
,
r
*
http
.
Request
)
{
if
err
:=
client
.
Ping
()
.
Err
();
err
!=
nil
{
http
.
Error
(
w
,
"Unable to connect to the redis master"
,
http
.
StatusInternalServerError
)
}
else
{
fmt
.
Fprintf
(
w
,
"All is well!"
)
}
})
err
:=
http
.
ListenAndServe
(
":8081"
,
nil
)
if
err
!=
nil
{
log
.
Printf
(
"Unable to start /healthz webserver"
)
}
}
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment