GCP LoadBalancer (pubsub)

Phase 1: Configure Your GCP Project

1. Enable GCP Load Balancer Logging

1. In the Google Cloud console, go to the Load Balancing page.

2. Click the name of your load balancer.

3. Click edit.

4. Click Backend Configuration.

5. Click edit next to your backend service.

6. In the Logging section, select the Enable logging checkbox.

7. To finish editing the backend service, click Update.

8. To finish editing the load balancer, click Update.

2. Create a Pub/Sub Topic

This topic will act as the destination for your logs.

1. Navigate to Pub/Sub > Topics.

2. Click Create Topic and give it an ID (e.g., loadbalancer-logs-topic).

1. Click Create.

   1. Create the Subscription:

      1. Go to the Subscriptions tab and click Create Subscription.

      

      
   2. Provide a Subscription ID, like central-loadbalancer-sub.

   

   c. From the dropdown menu, select the central-loadbalancer-logs-topic you just created.

d. Leave all other settings as their defaults and click Create.

3. Create a Log Sink

The sink will filter your loadbalancer logs and send them to the Pub/Sub topic.

1. Navigate to Logging > Log Router.

2. Click Create Sink.

1. Name the sink (e.g., loadbalancer-log-sink) and click Next.

2. For the Sink destination, select "Cloud Pub/Sub topic" as the sink service and choose the loadbalancer-logs-topic you just created. Click Next.

1. In the Build inclusion filter box, paste the following filter to select only loadbalancer logs:

logName:"logs/compute.googleapis.com%2Floadbalancer"

1. Click Create Sink.

4. Set Sink Permissions

After creating the sink, you must give it permission to publish to your topic.

1. Go to Log Router, select the sink which we have created earlier. Click on the three dots at the right and click on view sink details and copy the writer identity

1. The Log Router page will now show a Writer Identity for your new sink (an email like [e-mail]). Copy this email address.

2. Go back to Pub/Sub > Topics, click the checkbox next to loadbalancer-logs-topic, and open the PERMISSIONS tab in the info panel on the right.

3. Click ADD PRINCIPAL.

1. Paste the Writer Identity email as the new principal.

2. Assign it the role of Pub/Sub Publisher.

1. Click Save.

Phase 2: Create a Service Account for Filebeat

Filebeat needs its own identity to read the logs from GCP.

1. Navigate to IAM & Admin > Service Accounts.

1. Click Create Service Account, give it a name (e.g., filebeat-agent), and click Create and Continue.

1. In the "Grant access" step, assign the role of Pub/Sub Subscriber. Click Done.

1. Find the service account you just created, click on the three dots at the right and select Manage keys option.

2. Go to its KEYS tab, click ADD KEY, and select Create new key.

1. Choose JSON and click CREATE. A JSON key file will be downloaded to your computer.

2. Share the downloaded JSON file with Blusapphire along with project id ,topic , subscription name