Oracle Cloud Infrastructure - Terraform Setup

If you wish to set up the integration manually, return to the Get Started with Oracle Cloud Infrastructure page and choose your manual setup option.

We use Terraform to automate the OCI functions. The Terraform script provisions the central resources in the home region, as well as all necessary regional resources to allow the forwarding of logs to the central bucket.

Scope and Limitations

1. We have set the retention lock in the automation to 15 days (360h), because Terraform runs so fast that minor clock differences between your system and OCI's servers can cause a 14-day request to be rejected (OCI requires retention locks to be set at least 14 days in the future). Manual setup is slow enough that this is not an issue.
2. This automation supports complete rollback/teardown, however the central bucket and docker image cannot be removed via Terraform due to known limitations within OCI.

   • If you need to perform a rollback (such as to retry the setup process), make sure to use the ./deploy.sh rollback command to do so. This ensures all resources are removed in a way that does not create a conflict if you re-create resources of the same name.
   • The central bucket must be removed manually, unless you wish to use a different bucket name if/when you retry the setup.
   • The compartment can be removed via Terraform, but we recommend you remove it manually because compartment deletion may take several minutes to hours if done via Terraform.
   • We do not recommend you remove the docker image.

Prerequisites

1. Make sure there is a user from your organization with Admin-level cloud access to the relevant cloud environment.

   • The account you use could be your own OCI account, or it could be an administrator account.
2. Make sure you have an email address from your organization that can be used to associate with the Expel Machine User you will create.

   • This email will be used by you to generate the API credentials for Expel.
   • You should choose an organizational email you can access, like a shared security team mailbox or your work email.

Before You Begin

1. Locate all region identifiers for the regions you wish to onboard. You will need to specify these identifiers when you configure the automation resources.
2. Obtain the tenancy OCID and tenancy namespace for the OCI admin user (the one deploying the resources), which you will need when you configure the automation resources. You can find these values under Profile > Tenancy > Tenancy details (the namespace is labeled "Object storage namespace").

If you need additional help locating any of these values, see the Troubleshooting.

Step 1: Download the Automation Files

expel_oci_automation.zip

archive · 21KB

Open

The expel_oci_automation.zip file contains the Terraform configurations, function code, and deployment scripts you need to complete this guide.

Module	Contents	Purpose
terraform/central	Terraform configuration	Creates central/global resources (run once per tenant).
terraform/regional	Terraform configuration	Creates regional resources (run once per region).
function/	Dockerfile, func.py, func.yaml, requirements.txt	Function code deployed to each region.
config.env	Configuration file	User-provided values for the automation.
deploy.sh	Deployment script	Executes the setup process.

Step 2: Configure the Automation Resources

In this step, you will unzip the files you downloaded in the previous step so that you can update the config.env file. After you have finished and saved that update, you must re-zip the folder so that it can be used by the automation script in a later step.

Before you begin, make sure you have all region identifiers for the regions you wish to onboard, the admin user's tenancy OCID, tenancy namespace, and the organization email address you decided to associate with the Expel Machine User. If you need help locating any of these values, see the Troubleshooting.

1. Unzip the file you downloaded in Step 1.
2. In the top-level expel_oci_automation folder, locate the config.env file and open it in the text editor of your choice.
3. First, update the OCIR_USERNAME with an OCI account that has permissions to create all required resources.

   • The account you use here could be your own OCI account, or it could be an administrator account.
   • For native users, use the format NAMESPACE/EMAIL. Example: idhl111labc/[e-mail]
   • For federated users, use the format NAMESPACE/oracleidentitycloudservice/EMAIL. Example: idhl111labc/oracleidentitycloudservice/[e-mail]
4. Update the remaining values:

   • TENANCY_OCID - enter your tenancy OCID. If you need help locating this value, see the Troubleshooting.
   • EXPEL_AUDIT_USER_EMAIL - enter the org email you want to use for the Expel Machine User.
   • EXPEL_AUDIT_USER_FIRST_NAME - enter "Expel".
   • EXPEL_AUDIT_USER_LAST_NAME - enter "User".
   • REGIONS - enter the region identifiers for the regions you wish to onboard, using a csv string (example: “us-ashburn-1,us-chicago-1,us-phoenix-1”). If you need help locating the correct identifiers, see the Troubleshooting.
5. Save and close the updated file.
6. Re-zip the updated expel_oci_automation folder (you will need to use this folder in .zip format when you run the automation script in a later step).

Step 3: Generate an Auth Token

An auth token is required in order to deploy the automation script later in this guide.

1. Log in to OCI.
2. In the top right, select your username/email.

   
3. Select the Tokens and keys tab.
4. In the Auth tokens section, select Generate token.
5. Enter a description for the token, such as "Expel token".
6. Select Generate token.
7. Copy and save the generated token, as you will need it for a future step and it will not be available again.
8. Select Close.

Step 4: Upload the Automation File to OCI

You will use OCI's Cloud Shell to perform this step.

1. Make sure you are in your home region.
2. In the Developer Tools menu in the top right, select Cloud Shell. Wait for your machine to be created, and then continue to the next step.
3. In the top right of the Cloud Shell window, select Settings > Upload.
4. Drop or select the re-zipped, updated expel_oci_automation.zip file you configured in Step 2.
5. Select Upload.

Step 5: Deploy the Automation Script

Before you begin, make sure you have the auth token you generated in Step 3.

1. Still in the Cloud Shell window, use the following command to unzip the file you just uploaded:

unzip expel_oci_automation.zip

1. Switch to the file's directory:

cd expel_oci_automation

1. Grant execution permissions to the deployment script:

chmod +x ./deploy.sh

1. Execute the deployment script:

./deploy.sh

1. When prompted, enter the auth token you generated in Step 3.
2. All global and central resources will now be created, along with the regional resources in the selected regions.
3. When the process has finished, copy and save the following two values:

   • Stream OCID (this will be the Stream ID in Workbench)
   • Stream Endpoint (this will be the Stream endpoint in Workbench)

Step 6: Validate All Resources

Before continuing with the setup process, make sure all resources have been created and that there are no errors.

• You may search for the resources in whatever way is easiest for you. For more details, see Inspecting Resources in the OCI documentation.
• Search for each resource in the charts by name, as this is the name that was created by Terraform.

If you find any discrepancies or issues, follow the Troubleshooting steps to perform a correct rollback and try the setup process again or contact Support for help.

Global and Central Resources

#	Type	Resource	Name	Purpose
1	identity	compartment	expel_audit_compartment	Contains all Expel resources
2	identity	dynamic group	expel_audit_functions_dg	Grants permissions to OCI resources
3	identity	user group	expel_audit_functions_dg	Group for Expel machine user
4	identity	user	expel_audit_user	Expel machine user
5	identity	policy (compartment)	expel_audit_collection_policy	Dynamic group permissions
6	identity	policy (tenancy)	expel_audit_machine_policy	Machine user permissions
7	storage	bucket	expel_audit_central_bucket	Function target bucket (receives forwarded logs)
8	storage	lifecycle policy	expel_audit_lifecycle_policy	Set to auto-delete after 14 days
9	storage	retention rule	expel_audit_retention_rule	7-day retention lock
10	streaming	stream pool	expel_audit_stream_pool	Container for streams
11	streaming	stream	expel_audit_stream	Event notification queue
12	events	event rule	expel_audit_event_rule	Triggers on object create/update in central bucket

Regional Resources

These resources are created for each target region you onboard.

#	Type	Name	Purpose
1	service connector	expel_audit_sch_[RegionIdentifier]	Audit logs → regional function
2	application	expel_audit_func_app_[RegionIdentifier]	Container for the function
3	function	expel_audit_function_[RegionIdentifier]	Payload → central bucket
4	VCN	expel_audit_vcn_[RegionIdentifier]	Container for network components
5	service gateway	expel_audit_sgw_[RegionIdentifier]	Facilitate communication between OCI services
6	NAT gateway	expel_audit_natgw_[RegionIdentifier]	Facilitate communication between OCI services
7	security list	expel_audit_sl_[RegionIdentifier]	Control network traffic at the subnet level by defining firewall rules
8	routing table	expel_audit_rt_[RegionIdentifier]	Control where network traffic goes from a subnet
9	private subnet	expel_audit_subnet_[RegionIdentifier]	Divide a VCN into smaller network segments
10	registry repo	expel_audit_repo_[RegionIdentifier]	Store and manage container images
11	registry docker image	expel_audit_repo_[RegionIdentifier]/ expel_audit_function_[RegionIdentifier]:<version_tag>	Packaged container application stored inside OCIR
12	context	[RegionIdentifier]	Store and switch between OCI CLI configurations

Step 7: Obtain the Expel Credentials

These credentials will be used when you configure the security device in Workbench.

Before you begin, make sure you have logged out of OCI and looked for the activation email from Oracle.

Step 1: Generate the Credentials

1. Log in to OCI with the Expel Machine User email address you specified in the automation resources. You should have already received an activation email from Oracle.
2. In the top right, select the username/email.
3. Select the Tokens and keys tab.
4. Select Add API key.
5. Choose the Generate API key pair option.
6. Download the private key. This file will download as a .pem file.
7. Select Add.
8. In the Configuration file preview, use the Copy button to quickly copy and save the following values:

   • User (this will be the OCI user in Workbench)
   • Fingerprint (this will be the OCI key fingerprint in Workbench)
   • Tenancy (this will be the OCI user tenancy in Workbench)
   • Region (this will be the OCI region in Workbench)

Step 2: Base64 Encode the API Key

You must now format your private API key as a base64-encoded string, and save the output. This string is what you will use for the API key when you configure the security device in Workbench.

For security reasons, we do NOT recommend using an online Base64 encoding website.

Windows Systems

1. Open PowerShell.
2. Run the following command (drag your .pem file into the window after entering the first part):

[Convert]::ToBase64String([IO.File]::ReadAllBytes("<file location/path>"))

1. Copy and save the output as your base64-encoded API key.

Unix-Like Systems

1. Open Terminal.
2. Run the following command (drag your .pem file into the window after entering the first part):

base64 -i

1. Copy and save the output as your base64-encoded API key.

You may delete the .pem file after you have successfully set up the security device in Workbench. Please retain it until then. If you need to verify that your API key was encoded correctly, see the Troubleshooting.

Step 8: Add Oracle Cloud Infrastructure as a Security Device in Workbench

Before you begin, make sure you have the stream OCID and stream messages endpoint from Step 5, as well as the user, fingerprint, tenancy, region, and base64-encoded API key from Step 7.

1. Log in to Workbench.
2. In the side menu, navigate to Organization Settings > Security Devices. If you have multiple organizations, you must select the appropriate organization name from the list.
3. Select Add Security Device.
4. In the search box, type “Oracle” and then select the Oracle Cloud Infrastructure integration.
5. Complete the fields as follows:

   • Name - enter a name that might help you more easily identify this integration, such as “CompanyName OCI”; this name will display in Workbench under the Name column, and is a text string that you can filter on.
   • Location - enter the location of your integration, for example “cloud;” this is also a text string that you can filter on, so we recommend being consistent with location naming across your Expel integrations.
   • Stream ID - enter the stream ID you saved in Step 5.
   • Stream endpoint - enter the stream endpoint you saved in Step 5.
   • OCI user - enter the user value from the configuration file you saved in Step 7.
   • OCI key fingerprint - enter the fingerprint value from the configuration file you saved in Step 7.
   • OCI user tenancy - enter the tenancy value from the configuration file you saved in Step 7.
   • OCI region - enter the region identifier from the configuration file you saved in Step 7.
   • OCI user key - enter the base64-encoded string for the private API key, which you created in Step 7.
6. Select Save.
7. Select No thanks, I will not provide console access from the console access dropdown.

Your device should be created successfully within a few seconds. A few reminders:

• After your connection is healthy, it will take some time for your device to begin polling and receiving data.
• To check on the status, select the downward arrow for your device in the first column and choose View details.
• Polling will happen first; data will be received after that. You must refresh the page to see updates.
• If your device does not begin polling within 15 minutes, and does not begin receiving data within 30 minutes, contact our support team for help.
• To check if alerts are coming through, navigate to Dashboards > Alert Analysis. Scroll to the device you want to check, and select the Expel Alerts tab to reveal more alert information. It can take 36 to 72 hours for alerts to appear after setup, as we tune your device.

Troubleshooting

Device Connection Issues

The device does not connect successfully.

1. Make sure you selected the correct compartment when you deployed the stacks.
2. Verify that you downloaded the private API key and not the public one.
3. Check to be sure your API key was base64-encoded correctly (see below for help).
4. Make sure you pasted all values into the security device correctly, and that you removed any quotes. These values are case-sensitive.
5. Check your connector to make sure it is active.

You need to perform a rollback and retry the setup.

• Make sure to use the ./deploy.sh rollback command. This rollback method ensures all resources are removed in a way that does not create a conflict if you re-create resources of the same name.
• You will have to manually delete the central bucket or choose a different name when you retry the setup (known limitations within OCI prevent us from deleting this bucket via Terraform).
• The compartment can be removed via Terraform, but we recommend you remove it manually because compartment deletion may take several minutes to hours if done via Terraform.

You are having persistent issues after trying both of the above.

If issues persist after checking the above and/or retrying the setup, contact Support. Make sure to provide:

1. Your tenancy OCID and home region
2. The error message shown in Workbench
3. Screenshots of your central bucket and stream details

Region Identifier

When you set up the region-specific policy, you must use the correct region identifier. To determine the region identifier:

1. In the upper right, select Manage regions.
2. Locate your region to find its region identifier.

Tenancy OCID

You can find the tenancy OCID under Profile > Tenancy > Tenancy details.

Tenancy Namespace

You can find it on the same page as the Tenancy OCID, labeled as Object storage namespace.

Base64 Encoding

To verify your API key, you can decode your Base64 string and make sure the key matches the one in your .pem file.

Windows Systems

Open PowerShell and run:

[System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String('YOUR-BASE64-STRING'))

Unix-Like Systems

Open Terminal and run:

printf "%s" "YOUR-BASE64-STRING" | base64 -d