Skip to content

Discussion on adding custom resources types to manage tokens, users, groups, projects #395

Open
Open
Discussion on adding custom resources types to manage tokens, users, groups, projects#395
Assignees
alexjfisher
Labels
enhancementNew feature or requestneeds-feedbackFurther information is requested

Description

@tuxmea
tuxmea
opened on Nov 7, 2021

I am considering adding custom types and providers to manage gitlab internal configs.

What is the best way to interact from a custom type/provider with gitlab?

At the moment I see two ways:

  1. executing gitlab-rails runner
  2. communicate with api

gitlab-rails command

The gitlab-rails command can be run as root user. No further authentication required.
This allows to check existing and add new tokens, users, groups, projects.

But: the command is very slow.

api

Communication with api needs a token.
root token can be managed using gitlab-rails only.
other user tokens can be managed using the root token.

all other resource types will need an autorequire on the gitlab_token type.

But: where to store the root api token itself so it can be used by all other resource types?
In puppet cache dir?
or is it possible to pass a variable from one provider to another one without again running the gitlab-rails command?

Activity

  1. tuxmea
    added
    enhancementNew feature or request
    needs-feedbackFurther information is requested
    on Nov 7, 2021
  2. tuxmea
    assigned
    alexjfisher
    on Nov 7, 2021

  3. bastelfreak commented on Nov 8, 2021

    @bastelfreak
    bastelfreak
    on Nov 8, 2021
    Member

    @baurmatt can you take a look at well? Do you have any opinions on that topic?

  4. baurmatt commented on Nov 9, 2021

    @baurmatt
    baurmatt
    on Nov 9, 2021
    Contributor

    I've never really used the gitlab-rails runner command so I had to check it out on our system:

    $ time gitlab-rails runner 'puts Rails.env'
production

real	0m44.396s
user	0m36.290s
sys	0m5.594s

    This is our production Gitlab (8 CPU/32GB RAM). To be honest, I don't see how this can be successfully integrated. Even if all resource creation requests would share one gitlab-rails runner session this would be to long IMHO.

    So from my point of view the only option is the API - even thought that the API token handling might be a manual step.

    Regarding the storage location of the token: I don't like to put relevant things in the cache dir. I would prefer to just store it in /etc/gitlab with the restrictive permissions. This might be "ok" because an attacker who can access this file will also be able to just use gitlab-rails runner/console. From a design perspective it would be also nice to store the token in the PuppetDB because in a "classic" web setup this would be the place where state is persistent. Though I don't know how to implement this.

  5. tuxmea commented on Nov 9, 2021

    @tuxmea
    tuxmea
    on Nov 9, 2021
    MemberAuthor

    @baurmatt many thanks for your feedback.

    How about the following idea:
    gitlab_token type/provider check if the token file exists in /etc/gitlab/.
    If the file exists it does not do anything.
    If the file is missing, the token will be created using gitlab-rails runner command and written to the file.

    This will allow us to do a complete automated setup.

    Usage of the gitlab_token type will be optional.
    All other types/providers use the token file and API calls.

  6. baurmatt commented on Nov 9, 2021

    @baurmatt
    baurmatt
    on Nov 9, 2021
    Contributor

    Sounds good! Looking forward to the implementation! :)

  7. tuxmea commented on Nov 9, 2021

    @tuxmea
    tuxmea
    on Nov 9, 2021
    MemberAuthor

    @baurmatt
    List of types and properties:

    1. gitlab_token:
    • newproperty: user - String
    • newproperty: key - String
    • newproperty: scope - Array[String]
    1. gitlab_user
    • newproperty: username - String
    • newproperty: realname - String
    • newproperty: password (optional) - String
    • newproperty: confirmed - Boolean
    • newproperty email - String
    1. gitlab_group
    • newproperty: name - Strig
    • newproperty: member_role - Hash (user: role mapping)
    1. gitlab_project
    • newproperty: name - String
    • newproperty: group - String
    • newproperty: origin_url - Optional[String]

  8. baurmatt commented on Nov 9, 2021

    @baurmatt
    baurmatt
    on Nov 9, 2021
    Contributor

    @tuxmea You probably want email - String for gitlab_user as well.

    Beside this it looks like a good start! 👍

  9. alexjfisher commented on Nov 9, 2021

    @alexjfisher
    alexjfisher
    on Nov 9, 2021
    Member

    If using the API, could these types be used to manage projects/groups etc. on gitlab.com ? That might be useful to some people? (In which case you probably don't want to implement instances?? and you shouldn't expect gitlab-rails command to be available?)

    For the api settings file, (assuming you don't want to make the credentials parameters of each of the individual resources), you can do confine :exists => '/path/to/api_settings.yaml' in the providers and all types can also have

    autorequire(:file) do
 '/path/to/api_settings.yaml'
end

    I'd then probably provide a class that users can use to create this file. eg. the user would do something like...

    class { 'gitlab::api_settings':
  server => 'https://gitlab.com',
  token => Sensitive($my_api_token),
}

gitlab_project { 'my_project':
  group => 'some/group',
}

# etc.

    (With the gitlab_group type, dunno if groups need to know about their parent groups - perhaps so they can autorequire them?)

    Finally, I've used the gitlab gem before with good results. Maybe the provider should make use of this? (The module could offer to install it and the providers can confine themselves based on whether it's installed yet)

  10. tuxmea commented on Mar 1, 2022

    @tuxmea
    tuxmea
    on Mar 1, 2022
    MemberAuthor

    @alexjfisher sounds like a good idea. But: where can we receive the API token from? Or how to set the token explicitly?
    Using gitlab gem also came into my mind. But the same question: how to set or generate the API token?

  11. baurmatt commented on Mar 1, 2022

    @baurmatt
    baurmatt
    on Mar 1, 2022
    Contributor

    The token needs to be provided manually. If you don't want this, you could automate it with a local command as described here. This token would than be saved to e.g. /etc/gitlab/$user-token and loaded by the provider.

  12. alexjfisher commented on Mar 1, 2022

    @alexjfisher
    alexjfisher
    on Mar 1, 2022
    Member

    I guess for use with gitlab.com it would have to be manually provided, but if using the module to deploy your own gitlab installation, then being able to automatically generate it would be better.

  13. baurmatt commented on Mar 1, 2022

    @baurmatt
    baurmatt
    on Mar 1, 2022
    Contributor

    Is there an advantage to using these types/providers instead of the official Gitlab Terraform Provider on gitlab.com? While thinking about it... Is it worth developing custom types/provider at all? What would be the benefit?

  14. jay7x commented on Jul 31, 2025

    @jay7x
    jay7x
    on Jul 31, 2025
    Member

    Haha.. somehow I miss this issue before developing the gitlab_application_settings resource type here: #468. So I just put its settings to the /etc/puppetlabs/gitlab_provider.json file 🤷🏻‍♂️. Feel free to comment the PR (it's more or less done, just miss the SSL support).

    Biggest issue (not mentioned there) is a token requirement.. I know how to work it around in a VERY WEIRD way, but not sure I'd like to (HINT: look into the rake file in the PR).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestneeds-feedbackFurther information is requested

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions