Working with Puppet Resources
Introduction:
Puppet is a powerful configuration management tool used to automate the deployment and management of infrastructure. Central to Puppet's functionality are "resources" and "manifests." This document aims to provide a comprehensive understanding of these key concepts and their role in Puppet configuration.
Puppet Resources:
Definition:
A Puppet resource represents a piece of system configuration, such as a file, service, or package.
Resources are the building blocks of Puppet manifests and define the desired state of the system.
Working with Puppet Resources:
Resource declaration:
While defining a puppet resource inside a manifest file, we first set the resource type. This should be all in lower case. Then we set a title and key-value pair. This title and key-value pairs are enclosed in brace-brackets.
<resource-type> { <’resource-name’ or ‘title’>:
attribute => value,
}
To list all available puppet resource use command,
$ sudo puppet describe --list or,
$ sudo puppet resource --types
These are the types known to puppet:
In order to get documentation / more information about any of above resource, we can use command,
$ sudo puppet describe <resource_name>
While defining a resource in a manifest file, a ‘namevar’ can also be used to capture the variable name and use it instead of a title in resource declaration.
Now to start creating a manifest file and define the resources in it, let's take help of a command,
$ sudo puppet resource package ntp
we get below output on the screen.
package { 'ntp':
ensure => '1:4.2.8p12+dfsg-3ubuntu4.20.04.1',
mark => 'none',
provider => 'apt',
}
As we get above output after running the puppet resource command, we can redirect this output to a file with ‘.pp’ extension. And this can be the starting point for use to create the manifest. Off course we can write a resource definition in a new blank file as well.
$ sudo puppet resource package ntp > ntp_package.pp
Now let’s understand some important and common resource types with related attributes to define a resource in manifest.
1) Manage applications packages using package resource.
Manage packages. There is a basic dichotomy in package support right now: Some package types (such as yum and apt) can retrieve their own package files, while others (such as rpm and sun) cannot. For those package formats that cannot retrieve their own files, you can use the source parameter to point to the correct file.
Puppet will automatically guess the packaging format that you are using based on the platform you are on, but you can override it using the provider parameter; each provider defines what it requires in order to function, and you must meet those requirements to use a given provider.
package { 'package-name':
ensure => ‘installed’, # ‘absent’, ‘purged’, ‘latest’, ‘4.0’
name => ‘ntp’, # this can be used if we are not using actual package name as title.
provider => ‘apt’, # normally not required.
}
2) Manage Service using the service resource.
Manage running services. Service support unfortunately varies widely by platform --- some platforms have very little if any concept of a running service, and some have a very codified and powerful concept. Puppet's service support is usually capable of doing the right thing, but the more information you can provide, the better behavior you will get.
Here's an example to understand, how to declare service resource.
service { ‘service-name’:
ensure => ‘running’,
name => ‘ntpd’,
enable => true,
}
# above example, the value for ensure can be ‘stopped’, ‘false’, ‘true’
# name attribute is used if title is not same as service name, or the platform change is expected.
# false, true, manual, mask, delayed - no quotes as this holds a Boolean value.
3) Managing files on the node using the file resource.
Manages files, including their content, ownership, and permissions. The `file` type can manage normal files, directories, and symlinks; the type should be specified in the `ensure` attribute.
File contents can be managed directly with the `content` attribute, or downloaded from a remote source using the `source` attribute; the latter can also be used to recursively serve directories.
file {
owner => ‘root’,
group => ‘user’,
mode => ‘0644’,
ensure => ‘file’,
}
file { ‘/tmp/puppet’ :
ensure => ‘directory’,
}
file { ‘/tmp/puppet/file1’ : }
file { ‘/tmp/puppet/file2’ : }
file { ‘/tmp/puppet/file3’ :
mode => ‘0755’,
}
Using the file resource can also be combined with using variable as shown below.
In the first section, we define a variable as ntp_conf using $ sign.
$ntp_conf = “# managed by Puppet
server 192.168.33.101 iburst
driftfile /var/lib/ntp/drift”
file { ‘/etc/ntp.conf’:
ensure => ‘file’,
content => $ntp_conf,
}
Let's consider a sample manifest.pp file that contain all the above most commonly used resources like package, service and file defined in one single file.
$ntp_conf = “#Managed by puppet
server 192.168.33.101 iburst
driftfile /var/lib/ntp/drift”
package { ‘ntp’: }
# in this resource definition, if attributes like, ‘ensure’, provider’, are not defined then, by default attribute values will be defaulted and puppet will install and keep the service in running state.
file { ‘/etc/ntp.conf’:
ensure => ‘file’,
content => $ntp_conf,
owner => ‘root’,
group => ‘wheel’,
mode => ‘0644’,
}
service { ‘NTP_Service’:
ensure => ‘running’
enable => true,
name => ‘ntpd’,
}
…… in case of the platform is changed from RHEL to Debian, change the service name from ‘ntpd’ to ‘ntp’.
4) Manage users with user resource
This resource type uses the prescribed native tools for creating groups and generally uses POSIX APIs for retrieving information about them. 'user' resource is used for managing users on targeted nodes.
Here's an example of resource declaration for user resource.
user { ‘ganeshhp’:
ensure => ‘present’,
managehome => true,
groups => [ ‘sudo’, ‘users’],
password => pw_hash(‘Password1’,’sha512’,’salt’),
}
To create user group using group resource,
group { ‘admin’: }
### on most of the *nix platforms this resource can only create a group. Adding users to the group is to be managed using user resource for every user separately.
5) Managing /etc/hosts file using puppet host resource.
Using host resource, we can manage the hosts file entries, thereby, allowing dns name resolution.
In order to update the hosts file, we can use below code. This can be updated in the manifest.pp file as below.
Here's an example of defining the host resource to add an entry in the /etc/hosts file. in this example, the timeserver is the name of the node on which we want to make the changes.
host { ‘timeserver’:
ip => ‘192.168.33.110’,
host_aliases => ‘timeserver,tserver.example.com’
}
6) Executing arbitrary commands using ‘exec’ resource.
The ‘exec’ resource is used to execute / run arbitrary commands on the targeted host.
exec { ‘update_filesystem’:
command => ‘apt-get update -y’,
path => ‘/bin /usr/bin’,
cwd => ‘/var/tmp’,
provider => ‘shell’,
}
This example executes the apt-get update command to update the package database.
7) Using mounted file system with 'mount' resource
The ‘mount’ resource is used for managing mounted file system in targeted node. The actual behavior depends on the value of the 'ensure' parameter. Refresh: `mount` resources can respond to refresh events (via `notify`, `subscribe`, or the `~>` arrow). If a `mount` receives an event from another resource and its `ensure` attribute is set to `mounted`, Puppet will try to unmount then remount that filesystem.
mount { '/mnt/data':
ensure => 'mounted',
device => '/dev/sdb1',
fstype => 'ext4',
options => 'defaults',
}
Here, /mnt/data is mounted file system and attributes makes sure the device is available.
8) Manage schedules of cron job on *.nix system with ‘cron’ resource
This resource helps to installs and manages cron jobs.
cron { 'daily_cleanup':
command => '/usr/bin/cleanup_script.sh',
hour => 2,
minute => 30,
}
Here, in this example, a job is scheduled to perform a daily clean up on the targeted host to run the cleanup_script.sh at the defined time schedule.
Every cron resource created by Puppet requires a command and at least one periodic attribute (hour, minute, month, monthday, weekday, or special).
While the name of the cron job is not part of the actual job, the name is stored in a comment beginning with `# Puppet Name: `. These comments are used to match crontab entries created by Puppet with cron resources.
9) Similar to cron, we also have schedule resource.
Although ‘schedule resource‘ only allows you to run / schedule other resource run time / slot on the targeted host / node.
The puppet resource execution at the set slot / time can override the ‘runinterval’ setting which may have been scheduled to tun every hour / half hour etc.
schedule { 'maint':
range => '2 - 4',
period => daily,
repeat => 1,
}
10) Editing configuration files using Augeas tool.
The augeas resource allows you to interact with configuration files using the Augeas configuration editing tool.
Here are two examples that also explains the requirements for using Augeas tool.
Requires:
- [Augeas](http://www.augeas.net)
- The ruby-augeas bindings
@example Sample usage with a string:
augeas { "test1":
context => "/files/etc/sysconfig/firstboot",
changes => "set RUN_FIRSTBOOT YES",
onlyif => "match other_value size > 0",
}
@example Sample usage with an array and custom lenses:
augeas { "jboss_conf":
context => "/files",
changes => [
"set etc/jbossas/jbossas.conf/JBOSS_IP $ipaddress",
"set etc/jbossas/jbossas.conf/JAVA_HOME /usr",
],
load_path => "$/usr/share/jbossas/lenses",
}
Also, a simple way push changes in a file with notify function.
augeas { 'sysctl_conf':
changes => 'set net.ipv4.ip_forward 1',
notify => Service['network'],
}
This example sets the net.ipv4.ip_forward parameter in the sysctl.conf file.
11) Notify resource for sending an arbitrary message.
This resource helps to send an arbitrary message, specified as a string, to the agent run-time log. It's important to note that the notify resource type is not idempotent. As a result, notifications are shown as a change on every Puppet run.
So, the ‘notify’ resource is used to trigger notifications to other resources, such as,
notify { 'Rebooting now':
before => Exec['reboot_system'],
}
In this example a notification is send as ‘Rebooting now’ and will be displayed on the targeted host before the reboot action is initialized.
12) Ordering the application resources using stage resource.
The ‘stage’ resource is a resource type for creating new run stages. Once a stage is available, classes can be assigned to it by declaring them with the resource-like syntax and using the stage meta-parameter.
Note that new stages are not useful unless you also declare their order in relation to the default main stage.
The syntax for the resource declaration and configuration is as follows.
stage { 'pre':
before => Stage['main'],
}
class { 'apt-updates':
stage => 'pre',
}
.. here we define the stage with name as ‘pre’ and is defined to run prior to main stage, considering the main stage is already defined.
Now, let's understand some pre-defined stages. Below is the class defined to run as part of ‘pre’ stage. Note that individual resources can’t be assigned to run the stages, we can only set stages for classes.
Predefined stages:
Puppet has three pre-defined stages that can be used to control the order of resource application.
‘pre’: Applied first
‘main’: applied second (the default stage)
‘post’: Applied last
You can use these stages to ensure certain resources are applied at specific points during the Puppet run.
class my_module::pre {
notify { 'This is in the pre stage': }
}
class my_module::post {
notify { 'This is in the post stage': }
}
Custom stage:
We can also define custom stages to control the order of resource.
stage { 'before_main':
before => Stage['main'],
}
class my_module::before_main {
notify { 'This is before the main stage': }
}
Here the stage name ‘before_main’ can be anything else as well.
13) Manage authorized keys using ssh_authorized_key resource
The ssh_authorized_key resource type is used to manage SSH authorized keys for user accounts on targeted host machines. Currently only 2 types of keys are supported, ssh_rsa and rsa.
ssh_authorized_key { 'ganesh_hp:
ensure => present,
key => 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAA...',
}
In this example, the resource statement ensures that the specific SSH key is authorized for the ganesh_hp user on the targeted host machine.
If we want to refresh the keys and only specific keys are to be present on the host while removing any existing keys for the user, we can use the ‘purge_ssh_keys’ attribute to remove existing key and replace with specific key.
14) sshkey resource for setting up user authentication
In Puppet, the sshkey resource type is used to manage SSH authorized keys for user accounts. Here's an example of how to use the sshkey resource.
sshkey { 'ganeshhp_ssh_key':
ensure => present,
user => 'ganesh_hp',
type => 'ssh-rsa',
key => 'AAAAB3NzaC1yc2EAAAADAQABAA...',
options => ['from="192.168.1.1"', 'command="/usr/bin/echo Hello"'],
}
This sshkey resource ensures that the specified SSH key is present in the authorized keys file for the specified user. You can customize the parameters and options based on your specific requirements and security policies like the type of sshkey can be ssh-rsa or rsa.
15) Housekeeping of files in a directory / mounted drive location using tidy resource
The tidy resource is used for removing unwanted files based on specific criteria. Multiple criteria are OR'd together, so a file that is too large but is not old enough will get removed as per criteria and the folder / mount will get tidied and likewise we can apply permutation and combinations of such rules to keep the file storage tidy.
tidy { '/path/to/temp/files':
recurse => true,
matches => ['*.tmp', '*.temp']
age => '7d',
backup => false,
rmdirs => true,
ignore => ['.git', 'important_file.txt'],
force => true,
maxdepth => 2,
noop => false,
}
Here, we can apply the rule to the files with matching names or age or size of files. We can apply this to root directory or further down the path. Adjust the parameters based on your specific requirements and directory structure. While the files get removed we can even backup the files before removing. From the matching file extension, if we want to keep a file(s), we can add it to ignore list.
16) Manage Yum Repos on a Linux system using yumrepo resource
The yumrepo resource type in Puppet is used to manage YUM repository configurations on a system (RedHat family). You can use this resource type to define, modify, or remove YUM repositories.
Here's an example,
yumrepo { 'epel':
baseurl => 'https://download.fedoraproject.org/pub/epel/7/$basearch',
enabled => 1,
gpgcheck => 1,
gpgkey => 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7',
}
yumrepo { 'puppetlabs':
baseurl => 'https://yum.puppetlabs.com/el/7/products/$basearch',
enabled => 1,
gpgcheck => 1,
gpgkey => 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-puppetlabs',
descr => 'Puppet Labs Repository',
mirrorlist => 'https://yum.puppetlabs.com/el/7/products/mirrors',
}
In this example,
- In the first yumrepo block, defines a repository named 'epel'. It specifies the base URL for the repository, enables the repository (enabled => 1), performs GPG signature checking (gpgcheck => 1), and provides the path to the GPG key file.
- The second yumrepo block defines a repository named 'puppetlabs'. It includes additional settings such as a description (descr), a mirrorlist for repository mirrors, and a specific GPG key file.
17) Managing other resource using resources metatype
This is a metatype that can manage other resource types. Any metaparams specified here will be passed on to any generated resources, so you can purge unmanaged resources but set `noop` to true so the purging is only logged and does not actually happen.
This resource has some interesting parameters,
- name: This is the name of the resource type to manage.
- purge: Whether to purge unmanaged resources, when set to true. this will delete any resource that is not specified in configuration and is not 'autorequired' by any managed resource.
- unless_system_user: this keeps system users from being purged.
- unless_uid: This keeps specific UID and range of UIDs from being purged.
There are few other resources as well that are not listed and discussed in this document which are not so common like, ‘zfs’, ‘zone’, ‘zpool’, ‘selboolean’, ‘selmodule’.
One can refer to below link for puppet official documentation on resources. https://www.puppet.com/docs/puppet/7/lang_resources#lang_resources
Understanding Puppet Manifests:
Definition:
A Puppet manifest is a script written in Puppet's declarative language to define the desired state of a system.
Manifests consist of a collection of resource declarations that Puppet uses to enforce the specified configurations.
Components of a Manifest:
Nodes: Identify the target systems for applying configurations.
Classes: Group related resources for modularization and reusability.
Variables: Enable parameterization for flexible configuration.
While we write our manifest files, it is better to keep a habit of validating the code syntactically.
A simple way to create the manifest file with resource definition populated is with command,
$ sudo puppet resource user ganeshhp > user_manifest.pp
Or, we can make use of the PDK utility and command as shown below,
$ sudo pdk new module <module_name>
After we run above command we will get a module folder created with the name that we have provided in the command. Now, navigate to the module's folder. Inside that we can see the manifests folder. we can place the file that we have created in the first command in this folder location.
As we have seen in the earlier documents using the parser command to validate the syntax, is a good practice to follow. we use below command to validate the syntax.
$ sudo puppet parser validate user_manifest.pp
Now, let's look at the default locations where manifest files are present in the puppet code folder.
Site Manifest:
The manifest file where node statements are declared, resides in /etc/puppetlabs/code/environment/production/manifests folder. This is the folder location where the node and/or site manifest file is located. In the site (site.pp) or nodes (nodes.pp) manifest, classes are defined with it's module identity, e.g. webserver::apache .
Here's an example of site manifest,
# site.pp (located in manifests folder)
node 'webserver' {
include apache ## assign module class apache to the node
class { ‘webserver::apache’: } ## assign class apache from webserver module.
include webserver::user ## assign user class from webserver module
}
Module Manifest:
Manifest files are also part of the modules wherein the class is declared within which the resources are declared.
Here's an example of manifest file in a module, where we see class being declared within which we see resource declaration.
## modules/apache/manifests/init.pp
## declare main class in init.pp in apache module
class apache {
package { 'httpd':
ensure => installed,
}
service { 'httpd':
ensure => running,
}
}
Best Practices:
Modularity: Organize manifests into reusable modules for easier maintenance.
Version Control: Track changes to manifests using version control systems like Git.
Testing: Validate manifests in a testing environment before applying them to production systems.
Commentaires