doc patch

Author: netboxlabs-nvp

docs/configure-awx-aap.md

@@ -116,6 +116,36 @@ collections:
 
 Once you have built your Execution Environment, which is based on the default AWX Execution Environment, you can proceed with Proxmox automation in AWX.
 
+### Automating AWX NetBox/Proxmox Configuration
+
+There exists a convenience script in `netbox-proxmox-automation` that's called `./setup/configure_ansible_automation.py`, and it uses `awxkit` to manage objects in AWX.  It leverages the configuration that you wrote based on the sample configuration called `./conf.d/netbox_setup_objects.yml-sample`, and will save you an *immense* amount of time.
+
+Usage (inital setup):
+
+```
+shell$ cd /path/to/netbox-proxmox-automation/setup
+
+shell$ deactivate
+
+shell$ rm -rf venv
+
+shell$ python3 -m venv venv
+
+shell$ source venv/bin/activate
+
+shell$ pip install -r requirements.txt
+```
+
+Then make sure that you've made a copy of `../conf.d/netbox_setup_objects.yml-sample` and seasoned it to taste before proceeding.
+
+Usage (creation) of AWX objects that will be used for NetBox/Proxmox automation: `./configure_ansible_automation.py create --config /path/to/netbox-setup-objects.yml`
+
+Usage (removal) of *all* AWX objects that would be used for NetBox/Proxmox automation (*proceed with caution!*): `./configure_ansible_automation.py destroy --config /path/to/netbox-setup-objects.yml`
+
+### Manual AWX NetBox/Proxmox Configuration
+
+*This is not required if you have been able to run the convenience script, `./configure_ansible_automation.py`.*
+
 #### Add NetBox and Proxmox Credential Types to AWX
 
 Navigate to Administration > Credential Types in AWX, and create a credential type called 'NetBox Proxmox Creds'.

docs/configure-flask-application.md

@@ -1,11 +1,11 @@
 ## Configure Flask Application (Python)
 
-*You only need to do this configuration step if you intend to use the example Flask application to handle your Proxmox automation.*
+*You only need to do this configuration step if you intend to use the Flask application to handle your Proxmox automation.*
 
 Open a shell on your local system.  *Do not* run these commands as the 'root' user.  The following commands should run on MacOS, Linux, and UNIX-like systems; you will need to run these commands during initial installation or upgrades of `netbox-proxmox-automation`.
 
 ```
-shell$ cd /path/to/netbox-proxmox-automation/example-netbox-webhook-flask-app
+shell$ cd /path/to/netbox-proxmox-automation/netbox-event-driven-automation-flask-app
 
 shell$ deactivate # this will fail if there is no configured venv
 
@@ -28,7 +28,7 @@ shell$
 With each usage of `netbox-proxmox-automation`, make sure that you enter `venv` before running any Ansible commands.  Else this automation will not work.
 
 ```
-shell$ cd /path/to/netbox-proxmox-automation/example-netbox-webhook-flask-app
+shell$ cd /path/to/netbox-proxmox-automation/netbox-event-driven-automation-flask-app
 
 shell$ source venv/bin/activate
 
@@ -38,7 +38,7 @@ shell$ source venv/bin/activate
 When in `venv`, you will need to create `app_config.yml`.
 
 ```
-(venv) shell$ cd /path/to/netbox-proxmox-automation/example-netbox-webhook-flask-app
+(venv) shell$ cd /path/to/netbox-proxmox-automation/netbox-event-driven-automation-flask-app
 
 (venv) shell$ cp -pi app_config.yml-sample app_config.yml
 ```
@@ -58,5 +58,28 @@ Press CTRL+C to quit
  * Debugger PIN: XXX-XXX-XXX
 ```
 
-The above `flask` command will start the Flask application on port 8000 (or whatever you specify with the `-p` argument) and will bind on the IP address (or IP addresses) that were specified with the `-h` argument.  In this case, we used 0.0.0.0 with the `-h` argument, so the Flask application will listen on all interfaces.  The `--debug` argument indicates that we will run a single-threaded web service and that we will show output to stdout.  *You will want to use `gunicorn.py` or some other WSGI server to run the Flask application in production.*
+The above `flask` command will start the Flask application on port 8000 (or whatever you specify with the `-p` argument) and will bind on the IP address (or IP addresses) that were specified with the `-h` argument.  In this case, we used 0.0.0.0 with the `-h` argument, so the Flask application will listen on all interfaces.  The `--debug` argument indicates that we will run a single-threaded web service and that we will show output to stdout.
+
+*You will want to use `gunicorn.py` or some other WSGI server to run the Flask application in production.*  This is documented [here](https://flask.palletsprojects.com/en/stable/deploying/gunicorn/), but here's an annotated version of how to use gunicorn to run netbox-event-driven-automation-flask-app.
+
+1. Make sure that you are in the netbox-event-driven-automation-flask-app directory: `cd /path/to/netbox-proxmox-automation/netbox-event-driven-automation-flask-app`
+
+2. Leave `venv`: `deactivate`
+
+3. Remove existing `venv`: `rm -rf venv`
+
+4. Create new `venv`: `python3 -m venv venv`
+
+5. Enter new `venv`: `source venv/bin/activate`
+
+6. `pip` install requirements: `pip install -r requirements.txt`
+
+7. Run the application: `gunicorn -w 4 -b 0.0.0.0:9000 'app:app'`, or season this command to taste for how you want to bind, including your preferred port number for listening.
+
+You can then start using `netbox-event-driven-automation-flask-app` from NetBox!
+
+Granted, there are myriad ways to do this, including using NGINX to proxy connections to this Flask application, but that's an exercise that's left up to the reader for the time being.
+
+
+
 

docs/index.md

@@ -2,23 +2,36 @@
 
 [NetBox](https://github.com/netbox-community/netbox) is the world's most popular Network Source of Truth (NSoT) with tens of thousands of installations globally.  It is used extensively for documenting/modeling networks (network devices, virtual machines, etc), and also provides a great IPAM solution.  [Proxmox](https://www.proxmox.com/en/) is a freely available virtualization technology that allows you to deploy virtual machines (VMs) at scale, and perhaps in a clustered configuration.  Proxmox has approximately [900,000 hosts and more than 130,000 users in its open source community](https://www.proxmox.com/en/about/press-releases/proxmox-virtual-environment-8-1).
 
-When you think of the challenges of a widely used network documentation solution and a widely used virtualization technology, this implementation is an integration between virtual machine documentation (NetBox) and the automation of Proxmox virtual machine (VM) deployment and configuration.
+When you think of the challenges of a widely used network documentation solution and a widely used virtualization technology, this implementation is an integration between virtual machine documentation (NetBox) and the automation of Proxmox virtual machine (VM) and container (LXC) deployment and configuration.
 
-This automation handles creation, removal, and changes of/to Proxmox VMs.  The underlying automation uses [webhooks](https://demo.netbox.dev/static/docs/additional-features/webhooks/) and [event rules](https://netboxlabs.com/docs/netbox/en/stable/features/event-rules/) in NetBox.  When you induce a change in NetBox, this will set the desired VM state(s) in Proxmox.
+This automation handles creation, removal, and changes of/to Proxmox VMs *and* LXCs.  The underlying automation uses [webhooks](https://demo.netbox.dev/static/docs/additional-features/webhooks/) and [event rules](https://netboxlabs.com/docs/netbox/en/stable/features/event-rules/) in NetBox.  When you induce a change in NetBox, this will set the desired VM state(s) in Proxmox.
 
-When you create/update/delete VM objects in NetBox, the following will take place in Proxmox:
+When you create/update/delete VM objects (for Proxmox VM) in NetBox, the following will take place in Proxmox:
 
-- when you create a VM object in NetBox (name, status == Staged, chosen Proxmox VM template name), this will clone a VM in Proxmox of the same name, from the defined template
+- when you create a VM object in NetBox (name, status == Staged, VM Type == Virtual Machine, chosen Proxmox VM template name), this will clone a VM in Proxmox of the same name, from the defined template
 - when you add a SSH key to a NetBox VM object (status == Staged), a SSH key will be added to the VM settings in Proxmox
 - when you add a primary IP address to a NetBox VM object (status == Staged), this will update the VM settings in Proxmox for ipconfig0
 - when you add or resize VM disks (scsi0 - scsiN) for a NetBox VM object (status == Staged), this will:
-  - resize scsi0 on the Proxmox VM to the size that was defined in NetBox
-  - create scsi1 - scsiN on the Proxmox VM and set them to their specified sizes
-  - resize scsi1 - scsiN on the Proxmox VM and resize them to their specified sizes (*NOTE: Proxmox does not allow you to shrink disks!*)
+    - resize scsi0 on the Proxmox VM to the size that was defined in NetBox
+    - create scsi1 - scsiN on the Proxmox VM and set them to their specified sizes
+    - resize scsi1 - scsiN on the Proxmox VM and resize them to their specified sizes (*NOTE: Proxmox does not allow you to shrink disks!*)
 - when you remove a disk or disks from a NetBox VM object, this will remove the corresponding disks from the Proxmox VM (*NOTE: this does not include scsi0 as that is the OS disk*)
 
 Further:
 
 - when you set a VM's state to 'Active' in NetBox, this will start a VM in Proxmox
 - when you set a VM's state to 'Offline' in NetBox, this still stop a VM in Proxmox
 - when you remove a VM from NetBox, this will stop and remove a VM in Proxmox.
+
+When you create/update/delete VM objects (for Proxmox LXC) in NetBox, the following will take place in Proxmox:
+
+- when you create a VM object in NetBox (name, status == Staged, VM Type == LXC Container, chosen Proxmox VM template name, defined public SSH key), this will clone a LXC in Proxmox of the same name, from the defined template, and will also set the public SSH key (*NOTE: In LXC, you can only set the public SSH key once, and that's during the cloning process!*)
+- when you add a primary IP address to a NetBox VM object (status == Staged), this will update the VM settings in Proxmox for netif for the LXC
+- when you add or resize the VM disk (rootfs) for a NetBox VM object (status == Staged), this will:
+    - resize rootfs on the Proxmox LXC to the size that was defined in NetBox
+
+Further:
+
+- when you set a VM's state to 'Active' in NetBox, this will start a LXC in Proxmox
+- when you set a VM's state to 'Offline' in NetBox, this still stop a LXC in Proxmox
+- when you remove a VM from NetBox, this will stop and remove a LXC in Proxmox.

docs/netbox-customization.md

@@ -2,17 +2,11 @@
 
 You will need to do some customization to NetBox to define the underlying Proxmox VM configuration(s).  This section covers the custom field choices and custom fields that you'll need to create in NetBox -- in order for you to facilitate automation.
 
-### Custom Field Choices (for Proxmox VM templates and storage)
+### Custom Field Choices (for Proxmox VM and LXC automation)
 
-In the NetBox UI, you will need to create the following custom field choices
+You will need to perform some customization in NetBox before you can start automating Proxmox VMs and LXCs.
 
-1. `proxmox-node` will be used to correlate a Proxmox node to the Proxmox VM that you will provision
-2. `proxmox-vm-templates` will be used to correlate a Proxmox VM template with the Proxmox VM that you will provision
-3. `proxmox-vm-storage` will be used to correlate an underlying Proxmox VM storage volume with the Proxmox VM you will provision
-
-#### Automated NetBox Objects and Custom Fields Creation
-
-If you'd prefer to manually create the webhook and event rules in NetBox, you can skip to the next section.  Otherwise, proceed with the following to automate the creation of the webhook and event rules in NetBox.
+#### Automated NetBox Objects and the Creation of Custom Field Choices and Custom Fields
 
 `netbox-proxmox-automation` version 1.1.0 and newer ships with a convenience script, `netbox_setup_objects_and_custom_fields.py`, that when used alongside a configuration file of your choice, will greatly simplify this process.  In the case of AWX/Tower/AAP, `netbox_setup_objects_and_custom_fields.py` will query your AWX/Tower/AAP instance for project and template(s) information; this information will then be used to create the corresponding webhooks and event rules in NetBox.
 
@@ -56,92 +50,65 @@ shell$ source venv/bin/activate
 (venv) shell$ ./netbox_setup_objects_and_custom_fields.py --config /path/to/your/configuration.yml
 ```
 
-Then verify that everything has been created.  You can skip the rest of this document if so.
-
-
-If *not* using automation, in the NetBox UI, navigate to Customization > Custom Field Choices
-
-#### proxmox-node
-
-Create custom field choices for Proxmox VM Node(s).  When you click the '+' button, you will be presented with an Edit screen.  Fill the form as shown below.  Note that your choices will represent a list of Proxmox cluster nodes.  You will need to login to the Proxmox UI to get the list of Proxmox cluster nodes.
-
-![Screenshot of Proxmox VM Cluster Nodes Edit screen](./images/proxmox-cluster-nodes-edit.png)
-
-When you are done, your Custom Field Choices for Proxmox VM node(s) should look like this.
-
-![Screenshot of Proxmox VM Cluster Nodes View screen](./images/proxmox-cluster-nodes-saved.png)
-
-
-#### proxmox-vm-templates
-
-Create custom field choices for Proxmox VM Templates.  When you click the '+' button, you will be presented with an Edit screen.  Fill the form as shown below.  Note that your choices will have a (Proxmox) VMID to name-of-template mapping.  You will need to login to the Proxmox UI to get the VMID to name-of-template mappings.
-
-![Screenshot of Proxmox VM Templates Edit screen](./images/proxmox-vm-templates-edit.png)
-
-When you are done, your Custom Field Choices for Proxmox VM templates should look like this.
-
-![Screenshot of Proxmox VM Templates View screen](./images/proxmox-vm-templates-saved.png)
-
-
-#### proxmox-vm-storage
-
-Create custom field choices for Proxmox VM Storage.  When you click the '+' button, you will be presented with an Edit screen.  Fill the form as shown below.  Note that your choices will represent the name/value of each Proxmox storage volume.  You will need to login to the Proxmox UI to get a list of Proxmox storage volumes.
-
-![Screenshot of Proxmox VM Storage Edit screen](./images/proxmox-vm-storage-edit.png)
-
-When you are done, your Custom Field Choices for Proxmox VM storage should look like this.
-
-![Screenshot of Proxmox VM Storage View screen](./images/proxmox-vm-storage-saved.png)
-
-
-### NetBox Customization: Custom Fields (for Proxmox VMs)
-
-In the NetBox UI, you will need to create a series of custom fields, as noted below.
-
-1. `proxmox_node` will be used to correlate a Proxmox cluster node with the Proxmox VM that you want to create
-2. `proxmox_vm_template` will be used to correlate a Proxmox VM template with the Proxmox VM that you want to create
-3. `proxmox_vm_storage` will be used to correlate a Proxmox VM storage volume with the Proxmox VM that you want to create
-4. `proxmox_disk_storage_volume` will be used to correlate a Proxmox VM storage volume with each Proxmox VM disk that you want to create
-5. `proxmox_public_ssh_key` will be used to assign a public SSH key that you will use to login to a Proxmox VM
-6. `proxmox_vmid` will be used to document the Proxmox `vmid` that was created when the Proxmox VM was created
-
-In the NetBox UI, navigate to Customization > Custom Fields
-
-
-#### proxmox_node
-
-Create a custom field for Proxmox Node.  It will be called `proxmox_node`.  Here is what `proxmox_node` should look like after you've made your changes.
-
-![Screenshot of proxmox_node in NetBox UI](./images/proxmox-node.png)
-
-
-#### proxmox_vm_template
-
-Create a custom field for Proxmox VM template.  It will be called `proxmox_vm_template`.  Here is what `proxmox_vm_template` should look like after you've made your changes.
-
-![Screenshot of proxmox_vm_template in NetBox UI](./images/proxmox-vm-template.png)
-
-#### proxmox_vm_storage
-
-Create a custom field for Proxmox VM storage.  It will be called `proxmox_vm_storage`.  Here is what `proxmox_vm_storage` should look like after you've made your changes.
-
-![Screenshot of proxmox_vm_storage in NetBox UI](./images/proxmox-vm-storage.png)
-
-#### proxmox_disk_storage_volume
-
-Create a custom field for Proxmox disk storage volume.  It will be called `proxmox_disk_storage_volume`.  Here is what `proxmox_disk_storage_volume` should look like after you've made your changes.
-
-![Screenshot of proxmox_disk_storage_volume in NetBox UI](./images/proxmox-disk-storage-volume.png)
-
-#### proxmox_public_ssh_key
-
-Create a custom field for Proxmox VM public SSH key.  It will be called `proxmox_public_ssh_key`.  Here is what `proxmox_public_ssh_key` should look like after you've made your changes.
-
-![Screenshot of proxmox_public_ssh_key in NetBox UI](./images/proxmox-public-ssh-key.png)
-
-#### proxmox_vmid
-
-Create a custom field for Proxmox VMID.  It will be called `proxmox_vmid`.  Here is what `proxmox_vmid` should look like after you've made your changes.  *Note that `proxmox_vmid` is set automatically during the Proxmox VM provisioning process.  Any VMID that you specified will be discarded.*
+Then verify that everything has been created.  In the end, you should have Custom Fields for the following.
+
+  - proxmox_node:
+      - Object types: Virtual Machine
+      - Label: Proxmox node
+      - Group name: Proxmox (common)
+      - Type: Selection
+  - proxmox_vm_type:
+      - Object types: Virtual Machine
+      - Label: Proxmox VM Type
+      - Group name: Proxmox (common)
+      - Type: Selection
+  - proxmox_vmid:
+      - Object types: Virtual Machine
+      - Label: Proxmox Virtual machine ID (vmid)
+      - Group name: Proxmox (common)
+      - Type: Text
+  - proxmox_vm_storage:
+      - Object types: Virtual Machine
+      - Label: Proxmox VM Storage
+      - Group name: Proxmox (common)
+      - Type: Selection
+  - proxmox_public_ssh_key:
+      -  Object types: Virtual Machine
+      - Label: Proxmox public SSH key
+      - Group name: Proxmox (common)
+      - Type: Text (long)
+  - proxmox_lxc_templates:
+      - Object types: Virtual Machine
+      - Label: Proxmox LXC Templates
+      - Group name: Proxmox LXC
+      - Type: Selection
+  - proxmox_disk_storage_volume:
+      - Object types: Virtual Disk
+      - Label: Proxmox Disk Storage Volume
+      - Group name: Proxmox VM
+      - Type: Selection
+  - proxmox_vm_templates:
+      - Object types: Virtual Machine
+      - Label: Proxmox VM Templates
+      - Group name: Proxmox VM
+      - Type: Selection
+
+And you have Custom Field Choices for the following:
+
+  - proxmox-cluster-nodes: used by `proxmox_node` custom field
+      - Choices: available Proxmox nodes
+      - Default: first "discovered" Proxmox node
+  - proxmox-lxc-templates: used by `proxmox_lxc_templates` custom field
+      - Choices: "discovered" available Proxmox LXC images
+      - Default: first "discovered" Proxmox LXC image
+  - proxmox-vm-storage: used by `proxmox_vm_storage` custom field
+      - Choices: "discovered" Proxmox storage volumes
+      - Default: first "discovered" Proxmox storage volume
+  - proxmox-vm-templates: used by `proxmox_vm_templates` custom field
+      - Choices: "discovered" Proxmox VM templates
+      - Default: first "discovered" Proxmox VM template, based on lowest discovered vmid
+  - proxmox-vm-type: used by `proxmox_vm_type`
+      - Choices: Virtual Machine, LXC Container
+      - Default: Virtual Machine
 
-![Screenshot of proxmox_vmid in NetBox UI](./images/proxmox-vmid.png)