Ansible Playbook - How do I reverse engineer a running system?
-
I have a few VMs and PMs around the house that I'd setup over time and I'd now like to rebuild some, not to mention just simplify the whole lot.
How the hell do I get from a working system to an equivalent ansible playbook without many (MANY) iterations of trial & error - and potentially destroying the running system??
Ducking around didn't really show much so I'm either missing a concept / keyword, or, no-one does this.
Pointers?
TIA
@Cyber If you have some old wiki notes on how the system was setup originallythen it night be easier to ignore the current system and translate the wiki instructions into ansible. Still manual, but easier than reverse engineering. Another thing you can look at is bash history. Apart from backing up/cloning the system before you start I would also get a copy of the bash history for the various users and add it to a wiki or issue too. It will be useful.
-
You will need many iterations of trial and error. No way.
You can speed up testing your playbook by using Molecule or something similar. Don't touch your working VMs until you get a service (role) set up correctly in your test environment. If you need to set up multiple services in a single VM, you can automate their deployment sequentially, of course.
P. S. I don't like Ansible and won't recommend it because it is full of bugs and non-obvious behavior. However I didn't investigate alternatives and can't suggest a better one.
Could you elaborate a little bit about "full of bugs" and "non-obvious behaviour"?
I use Ansible at work for a couple of years already and never encountered anything like that. (I have about 10 playbooks, about 30 roles, about 20 linux servers that I administer) -
I have a few VMs and PMs around the house that I'd setup over time and I'd now like to rebuild some, not to mention just simplify the whole lot.
How the hell do I get from a working system to an equivalent ansible playbook without many (MANY) iterations of trial & error - and potentially destroying the running system??
Ducking around didn't really show much so I'm either missing a concept / keyword, or, no-one does this.
Pointers?
TIA
What I did to get rid of my mess, was to containerize service after service using podman. I mount all volumes in a unified location and define all containers as quadlets (systemd services). My backup therefore consists of the base directory where all my container volumes live in subdirectories and the directory with the systemd units for the quadlets.
That way I was able to slowly unify my setup without risking to break all at once. Plus, I can easily replicate it on any server that has podman.
-
Could you elaborate a little bit about "full of bugs" and "non-obvious behaviour"?
I use Ansible at work for a couple of years already and never encountered anything like that. (I have about 10 playbooks, about 30 roles, about 20 linux servers that I administer)Same question. But with 100s of playbooks, and thousands of servers. This feels like someone had a bad experience with their first 30 minutes of ansible and gave up before looking at the command reference.
-
What I did to get rid of my mess, was to containerize service after service using podman. I mount all volumes in a unified location and define all containers as quadlets (systemd services). My backup therefore consists of the base directory where all my container volumes live in subdirectories and the directory with the systemd units for the quadlets.
That way I was able to slowly unify my setup without risking to break all at once. Plus, I can easily replicate it on any server that has podman.
Do you have a GitHub repo? As I am building my system like this and was thinking of exactly using Podman and quadlets.
-
Could you elaborate a little bit about "full of bugs" and "non-obvious behaviour"?
I use Ansible at work for a couple of years already and never encountered anything like that. (I have about 10 playbooks, about 30 roles, about 20 linux servers that I administer)No, I can't. I use it only occasionally, so I don't remember everything. But many times configurations didn't work as described in documentation and I had to find a different way to achieve a required result. Sometimes this behavior changed from release to release. This thing doesn't seem something that I can rely on. But we use it in our company many years, so switch to another tool would be painful.
-
Same question. But with 100s of playbooks, and thousands of servers. This feels like someone had a bad experience with their first 30 minutes of ansible and gave up before looking at the command reference.
No, not 30 minutes. For the first time I spent couple of weeks just for reading documentation and experiments. It was about 8 years ago IIRC. But since that time when I need something more complex than install a package or copy a file, I feel myself like a 30-minutes user because it does not work as I expect.
-
No, not 30 minutes. For the first time I spent couple of weeks just for reading documentation and experiments. It was about 8 years ago IIRC. But since that time when I need something more complex than install a package or copy a file, I feel myself like a 30-minutes user because it does not work as I expect.
Fair enough. I honestly didn't mean this as an insult. I have seen the same type of review from people who join teams that I'm on when they get told about ansible.
It certainly isn't perfect. And there was a period of time about 5 years ago where a lot of change was happening at once.
Thanks for sharing your opinion
-
Do you have a GitHub repo? As I am building my system like this and was thinking of exactly using Podman and quadlets.
No, I keep that private to minimize the information I leak about what I host, sorry. (I also don't do git-ops for my server; I back the mentioned directories up via kopia so in case of recovery I just restore the last working state of data+config. I don't have much need to version the configs.)
-
I have a few VMs and PMs around the house that I'd setup over time and I'd now like to rebuild some, not to mention just simplify the whole lot.
How the hell do I get from a working system to an equivalent ansible playbook without many (MANY) iterations of trial & error - and potentially destroying the running system??
Ducking around didn't really show much so I'm either missing a concept / keyword, or, no-one does this.
Pointers?
TIA
Ducking around
giggle
-
@Cyber If you have some old wiki notes on how the system was setup originallythen it night be easier to ignore the current system and translate the wiki instructions into ansible. Still manual, but easier than reverse engineering. Another thing you can look at is bash history. Apart from backing up/cloning the system before you start I would also get a copy of the bash history for the various users and add it to a wiki or issue too. It will be useful.
Yeah... notes... they started about 50% of the way through building the system.
Now, my notes are great, but some of these devices are ~10 years old.
But, yep, I totally agree, notes are a damn good thing to have.
Not thought about bash history though, interesting point, but I think that only goes back a short duration?
-
@Cyber Yeah it's gonna be pretty manual as others have mentioned. Some areas to look at:
- Filesystem provisioning, mounts, etc.
- Packages
- Users, groups
- Time zone, locale language, time format etc.
- /etc/
- /root/ and /home/
- SSH settings
- Services
- Cron jobs/systemd timersThere is a bit of overlap between some of those categories. Some bits are going to see more or less use on VMs vs physical. And remember that in ansible there are built in modules for a lot of functionality.
Hadn't thought about all the locale, etc.... good point, thanks
-
I would copy the existing system onto a new system:
- Update system to the latest packages
- Create a new base system using the same distro
- Check which packages are not on the new system, add them to your playbook
- Install packages on new system
- This will take some time. Run a find of all files and pass them to md5sum or sha512sum to get a list of files with their checksum. Compare the list from the old system to the new system.
- Update your playbook with these findings. Template is probably the way to go, Lineinfile might be good as well, use copy if nothimg else works.
- Check firewall settings and update your playbook.
Anyhow this will take some iterations, but while you have a copy of your ‘production’ system, you can test on your ‘test’ machine until you have the same functionality.
oh the find with the hash sum is good advice! I would have done this but manually, maybe with the double commander sync dirs tool.
but also, for configs this might be the best time to move your custom config to ordered dropin files for all things that support it.
-
I have a few VMs and PMs around the house that I'd setup over time and I'd now like to rebuild some, not to mention just simplify the whole lot.
How the hell do I get from a working system to an equivalent ansible playbook without many (MANY) iterations of trial & error - and potentially destroying the running system??
Ducking around didn't really show much so I'm either missing a concept / keyword, or, no-one does this.
Pointers?
TIA
I went through this about 6 months ago.
Just build playbooks from basic to specific. I did so in three parts:
- Container creation
- Basic settings common to all my hosts
- Specific service config & software
Ansible assumes you have a hierarchy of roles to apply for each service, so layering playbooks this way should help
