Cloud Log Forwarding

Office-365 / Azure AD Integration

Log forwarding via Azure Event Hub

This page showcases multiple methods & instructions that guides you in setting up O365/Azure AD Integration and Configuration.

Overview

Before shipping logs from Azure Event Hubs to BluSapphire, you must stream the logs you want to be analyzed to Azure Event Hubs. Make sure to stream your logs to the same event hub.

• Azure Monitor collects logs for most Microsoft Azure services and streams the data to Azure Event Hubs.

• Azure Event Hub is a data streaming platform and event ingestion service for Microsoft Azure. Event Hub streams the logs collected by Azure Monitor.

• With further integration between BluSapphire SIEM and Azure Event Hubs, logs can be received and processed from an event hub.

Type of logs sent through Azure Event Hubs includes, but not limited to the following:

• Azure Active Directory (AD) logs, including audit logs and sign-in logs

• Azure Monitor logs

• Azure Application Gateway logs

• Azure SQL Database logs

• Microsoft Defender Advanced Threat Protection (ATP) logs

• Microsoft Intune logs

Configure different log stream to Azure Event Hub

To configure streaming for the logs you want to collect into event hubs, please follow Microsoft documentation:

1. Azure AD logs: See instructions from Microsoft on streaming Azure Active Directory Logs to an Azure Event Hub.

2. Azure Monitor logs: See instructions from Microsoft on creating Diagnostic Settings to Send Logs, make sure to select Event Hub as the destination to stream logs to Event Hub.

3. Azure Application Gateway logs: See instructions from Microsoft on enabling Logging for Application Gateway, make sure to select Event Hub as the destination to stream logs to Event Hub.

4. Azure SQL Database logs: See instructions from Microsoft on setting up auditing for your database, make sure to select Event Hub as the destination.

5. Microsoft Defender ATP logs: See instructions from Microsoft on configuring Microsoft Defender ATP to stream Advanced Hunting events to your Azure Event Hubs.

6. Microsoft Intune logs: See instructions from Microsoft on sending log data to event hubs, in Intune.

As the document progresses, create an event hub using the Azure portal.

Method-1: Azure Configuration (Create Event Hub Namespace & Event Hub)

Step01 - Creating Resource Group

A resource group is a logical collection of Azure resources. All resources are deployed and managed in a resource group. To create a resource group:

1) Login to the Azure portal: https://portal.azure.com/

2) Click Resource groups in the left navigation pane and Click Create on Resource groups page.

3) For Subscription, select the name of the Azure subscription in which you want to create the resource group.

4) Enter a unique name for the resource group, The system immediately checks to see if the name is available in the currently selected Azure subscription.

5) Select a Region for the resource group. Click Review + Create. Click Create again on next page.

Step02 - Creating Event Hub Namespace

An Event Hub namespace provides a unique scoping container, referenced by its fully qualified domain name, in which you create one or more event hubs. To create a namespace in your resource group using the portal, complete the following steps:

1) In the Azure portal, click Create a resource at the top left of the screen.

2) Search for Event Hubs, and then click the Create button in the Event Hubs page.

3) On the Create namespace page, complete the following steps:

3.1 Select the subscription in which you want to create the namespace.

3.2 Select the Resource group for the namespace.

3.3 Enter a name for the namespace. The system immediately checks to see if the name is available.

3.4 Select a location for the namespace and choose the pricing tier “Standard" – required for additional, custom consumer groups other than $Default group.

3.5 Click Review + Create and then Create. This will take few minutes for the system to fully provision the resources.

3.6 Refresh the Event Hubs page to see the event hub namespace. You can check the status of the event hub creation in the alerts. Select the namespace. You see the home page for your Event Hubs Namespace in the portal.

Step03 - Creating an Event Hub

To Create an Event Hub within the namespace, follow these steps:

1) In the Event Hubs Namespace page, click Event Hubs in the left menu.

2) At the top of the window, click + Event Hub.

3) Choose an appropriate name for your event hub, then click Create

4) After the event hub is created, you see it in the list of event hubs.

Note: This step is not required for all types of logs. Some log sources automatically create their own event hub like insights-logs-auditlogs, insights-logs-signinlogs, insights-activity- logs, others, within the namespace you created above. We recommend creating an Event Hub for sake of clarity.

Step04 - Configure an Event Hub Namespace

To Generate Shared Access Policy for Event Hub Namespace you’ve created

1) Select event hub namespace and go to Shared access policies, and then click +Add. Enter the Policy name, check the Manage box, and then click Create.

2) Click on the Shared Access policies you just created, make a note of the connection string primary key field for later use.

3) BluSapphire collector requires the SAS Policy name (created in Step04 - 1) and the connection string primary key when creating the credential in “blusapphire-sas". Copy the connection string primary key and policy name to a text editor for later use.

Note: When the event hub namespace is created, Azure will also create a default Shared Access Policy named RootManageSharedAcessKey.

4) Select an event hub namespace and go to respective Event Hub.

5) From the selected event hub, go to Consumer group within that event hub. Click +Consumer group to create a new consumer group.

Note: Please ensure you select Standard tier while creating Event Hub Namespace, this allows creating multiple Consumer Groups.

Information required to configure BluSapphire Collector

Following information is to be shared and required to setup BluSapphire Collector: These details can be obtained from steps performed earlier from Step01 to Step04

Configure different log stream to Azure Event Hub

To configure streaming for the logs you want to collect into event hubs, please follow Microsoft documentation:

1. Azure AD logs: See instructions from Microsoft on streaming Azure Active Directory Logs to an Azure Event Hub.

2. Azure Monitor logs: See instructions from Microsoft on creating Diagnostic Settings to Send Logs, make sure to select Event Hub as the destination to stream logs to Event Hub.

3. Azure Application Gateway logs: See instructions from Microsoft on enabling Logging for Application Gateway, make sure to select Event Hub as the destination to stream logs to Event Hub.

4. Azure SQL Database logs: See instructions from Microsoft on setting up auditing for your database, make sure to select Event Hub as the destination.

5. Microsoft Defender ATP logs: See instructions from Microsoft on configuring Microsoft Defender ATP to stream Advanced Hunting events to your Azure Event Hubs.

6. Microsoft Intune logs: See instructions from Microsoft on sending log data to event hubs, in Intune.

Note: While configuring each of the above log stream, make sure you choose “Stream to an event-hub” in the “destination details” and select the correct Subscription, Event Hub Namespace, Event Hub, Policy name.

Method-2: Log via Azure Event Hub

Subscription of "Azure Event Hub" in Azure cloud, subscribe to all the event-types that is needed to be ingested into BluSapphire platform.

Create an event hub using Azure portal

Azure Event Hubs is a Big Data streaming platform and event ingestion service, capable of receiving and processing millions of events per second. Event Hubs can process, and store events, data, or telemetry produced by distributed software and devices. Data sent to an event hub can be transformed and stored using any real-time analytics provider or batching/ storage adapters.

As the document progresses, create an event hub using the Azure portal.

Create a resource group

A resource group is a logical collection of Azure resources. All resources are deployed and managed in a resource group. To create a resource group:

1. Sign in to the Azure portal.

2. In the left navigation, click Resource groups. Then click Add.

3. For Subscription, select the name of the Azure subscription in which you want to create the resource group.

4. Type a unique name for the resource group. The system immediately checks to see if the name is available in the currently selected Azure subscription.

5. Select a region for the resource group.

6. Select Review + Create.

7. On the Review + Create page, select Create.

Create an Event Hubs namespace

An Event Hubs namespace provides a unique scoping container, referenced by its fully qualified domain name, in which you create one or more event hubs. To create a namespace in your resource group using the portal, do the following actions:

1) In the Azure portal and click Create a resource at the top left of the screen.

2) Select All services in the left menu and select star (*) next to Event Hubs in the Analytics category. Confirm that Event Hubs is added to FAVORITES in the left navigational menu.

3) Select Event Hubs under FAVORITES in the left navigational menu, and select Add on the toolbar.

4) On the Create namespace page, take the following steps:

5) Select the subscription in which you want to create the namespace.

6) Select the resource group you created in the previous step.

7) Enter a name for the namespace. The system immediately checks to see if the name is available.

8) Select a location for the namespace.

9) Choose the pricing tier (Basic or Standard).

10) Leave the throughput units settings as it is. To learn about throughput units, see Event Hubs scalability.

11) Select Review + Create at the bottom of the page.

12) On the Review + Create page, review the settings, and select Create. Wait for the deployment to complete.

13) On the Deployment page, select Go to resource to navigate to the page for your namespace.

14) Confirm that you see the Event Hubs Namespace page similar to the following example:

Create an event hub

To create an event hub within the namespace, do the following actions:

1) On the Event Hubs Namespace page, select Event Hubs in the left menu.

2) At the top of the window, click + Event Hub.

3) Type a name for your event hub, then click Create.

4) You can check the status of the event hub creation in alerts. After the event hub is created, you see it in the list of event hubs as shown in the following image:

Enabling Log Service onto Event Hubs:

Azure offers multiple subscription opportunities based on the business requirements.

O365 component provides multiple logging capabilities such as Audit logs, Message tracing, SharePoint logs, Etc. If a log source is to be onboarded onto BluSapphire, the log source to be enabled over event hubs before forwarding.

Customer/ Partner to decide on what events should be consumed by the SIEM, since within Office365 suite, there are multiple event-types which may be consumed, few examples to quote are audit logs, sign-in logs, metrics Etc.

Once the customer decides on the event-types which are to be ingested, customer needs to connect those event-types to the EventHub according to the instructions.

Let us take an example of onboarding Azure AD logs via Event Hubs:

Prerequisites

• An Azure AD tenant.

• A user who is a global administrator or security administrator for the Azure AD tenant.

• An Event Hubs namespace and an event hub in your Azure subscription. Stream logs to an event hub

1. Sign in to the Azure portal.

2. Select Azure Active Directory > Monitoring > Audit logs.

3. Select Export Settings.

4. In the Diagnostics settings pane, do either of the following:

a) To change existing settings, select Edit setting.

b) To add new settings, select Add diagnostics setting.

c) You can have up to three settings.

Select the Stream to an event hub check box, and then select Event Hub/Configure.

5) Select the Azure subscription and Event Hubs namespace that you want to route the logs to. The subscription and Event Hubs namespace must both be associated with the Azure AD tenant that the logs stream from. You can also specify an event hub within the Event Hubs namespace to which logs should be sent. If no event hub is specified, an event hub is created in the namespace with the default name insights-logs-audit.

6) Select OK to exit the event hub configuration.

7) Do either or both of the following:

a) To send audit logs to the event hub, select the AuditLogs check box.

b) To send sign-in logs to the event hub, select the SignInLogs check box.

8) Select Save to save the setting.

9) After about 15 minutes, verify that events are displayed in your event hub. To do so, go to the event hub from the portal and verify that the incoming messages count is greater than zero.

Access data from your event hub

At this point our collector can connect to the EventHub and pull the corresponding events. There is some access Key mojo which is need to be configured and do before our collector will get access to the EventHub but that is based on the external address of the EventHub and we can instruct the customer in that regards.

Method-3: O365 Log forwarding Via Graph API

This section provides steps on configuring Microsoft O365 for shipping log, following data can be collected and audited.

• Azure Active Directory

• Exchange

• SharePoint

• DLP

• General (Other Workloads)

Prerequisites

• An active subscription for Microsoft 365 with administrative privileges

• Unified Audit Logging enabled within M365’s Security and Audit Center.

• Machine with Filebeat (version 8 or greater) installed, to ship logs to BluSapphire Gateway Collector, preferably a windows machine.

Configuration

Step01 – Register a new application in Azure AD

1) Navigate to portal.azure.com and sign-in with an administrative account.

2) Open the pop-out menu from the top left corner of the web page and click on Microsoft Entra ID.

3) Under the Manage section on the left-hand side of the page, select App registrations > New registration

4) Provide Application Name, and then adjust options as necessary. The default settings should be fine if working with a single M365 tenant without a URI proxy

Note: Take note of the Application (client) ID and the Directory (tenant) ID. These will be used later while configuring Filebeat O365 Service Module.

Step02 – Create a client secret for your application

1) From newly created application overview page, navigate to Certificates & secrets. A client secret will be needed for your Filebeat configuration in order to pull data from the management API.

2) Select + New client secret and provide a description of your choosing and the certificate expiry.

3) Make note of the Value of the new client secret, as this will only be shown once.

Step03 – Configure management API permissions

1) To allow interaction with the management API, configure app permissions by navigating to API Permissions on the left hand side of the page.

2) Select + Add a permission and scroll down to find the “Office 365 Management APIs” widget.

3) From the Application permissions tab, enable the ActivityFeed.Read, ActivityFeed.ReadDlp and ServiceHealth.Read permissions.

Step04 – Configuring Filebeat for O365

1) Move on to machine hosting filebeat service, and navigate to the modules.d directory, open the “o365.yml.disabled” file.

2) Content of the file should be something like below, change the enabled parameter value to "true" and update "var.application_id, var.tenants, var.client_secret" parameter values with “Application ID, TenantID, TenantName (from Step01 – 4), & Client Secret (from Step02 – 3)” generated earlier.

// o365.yml

# Module: o365
# Docs: https://www.elastic.co/guide/en/beats/filebeat/7.x/filebeat-module-o365.html
- module: o365
audit:
enabled: true
# Set the application_id (also known as client ID):
var.application_id: <<Application ID>>   // Provide 'Application ID' created earlier
# Configure the tenants to monitor:
# Use the tenant ID (also known as directory ID) and the domain name.
# var.tenants:
# - id: "tenant_id_1"
# name: "mydomain.onmicrosoft.com"
# - id: "tenant_id_2"
# name: "mycompany.com"
var.tenants:
- id: <<TenantID>>      // Provide 'TenantID' created earlier
name: <<TenantName>>    // Provide 'TenantName' created earlier
# List of content-types to fetch. By default all known content-types
# are retrieved:
# var.content_type:
# - "Audit.AzureActiveDirectory"
# - "Audit.Exchange"
# - "Audit.SharePoint"
# - "Audit.General"
# - "DLP.All"
# Use the following settings to enable certificate-based authentication:
#var.certificate: "C:\ProgramData\filebeat\COMODomainValidationSecureServerCA.crt"
# var.key: "/path/to/private_key.pem"
# var.key_passphrase: "myPrivateKeyPassword"
# Client-secret based authentication:
# Comment the following line if using certificate authentication.
var.client_secret: <<Client Secret>>  // Provide 'ClientSecret' generated earlier
# Advanced settings, use with care:
# var.api:
# # Settings for custom endpoints:
# authentication_endpoint: "https://login.microsoftonline.us/"
# resource: "https://manage.office365.us"
#
# max_retention: 168h
# max_requests_per_minute: 2000
# poll_interval: 3m

3) Save “o365.yml.disabled” file and rename it as “o365.yml”

4) Open main Filebeat configuration file "filebeat.yml" and update “output – logstash – hosts” config parameter in "filebeat.yml" configuration file with BluSapphire Gateway Collector IP & Port (As shared by deployment team).

// Filebeat.yml
# Filebeat Configuration for O365 module
filebeat.config:
modules:
enabled: true
path: modules.d/*.yml
fields:
codec: plain
type: o365
fields_under_root: true
#For version 6.x and lower uncomment the line below and remove the line after it
#filebeat.registry_file: 'C:\ProgramData\Filebeat'
filebeat.registry.path: 'C:\ProgramData\filebeat'
#The following processors are to ensure compatibility with version 7
processors:
- rename:
fields:
- from: "agent"
to: "beat_agent"
ignore_missing: true
- rename:
fields:
- from: "log.file.path"
to: "source"
ignore_missing: true
############################# Output ##########################################
output:
logstash:
hosts: ["<<LISTENER-HOST>>:<PORT>"]  // Provide BluSapphire Collector IP & PORT

5) Save the “filebeat.yml” configuration file, restart Filebeat service to load the new configuration.

Note: Having all the above configuration steps performed, and Filebeat service is running and It may take some time to get the logs populated in the BluSapphire portal.

Method-4: Log forwarding via Azure API

What is required for BluSapphire?

• tenant_domain:

• client_secret

• client_id: aka application id (GUID)

• directory_id: aka tenant id (GUID)

the following content types will be pulled from the API

Kindly follow the URL for the required content types: https://docs.microsoft.com/en-us/office/office-365-management-api/office-365-management-activity-api-reference#working-with-the-office-365-management-activity-api

Permissions required on the following content type based on the choice of customer/ requirement.

• Audit.AzureActiveDirectory

• Audit.Exchange

• Audit.SharePoint

• Audit.General

Getting started with Office 365 Management APIs

When you create an application that needs access to secured services like the Office 365 Management APIs, you need to provide a way to let the service know if your application has rights to access it. The Office 365 Management APIs use Azure AD to provide authentication services that you can use to grant rights for your application to access them.

There are four key steps:

1) Register your application in Azure AD. To allow your application access to the Office 365 Management APIs, you need to register your application in Azure AD. This allows you to establish an identity for your application and specify the permission levels it needs to access the APIs.

2) Get Office 365 tenant admin consent. An Office 365 tenant admin must explicitly grant consent to allow your application to access their tenant data by means of the Office 365 Management APIs. The consent process is a browser-based experience that requires the tenant admin to sign into the Azure AD consent UI and review the access permissions that your application is requesting, and then either grant or deny the request. After consent is granted, the UI redirects the user back to your application with an authorization code in the URL. Your application makes a service-to-service call to Azure AD to exchange this authorization code for an access token, which contains information about both the tenant admin and your application. The tenant ID must be extracted from the access token and stored for future use.

3) Request access tokens from Azure AD. Using your application's credentials as configured in Azure AD, your application requests additional access tokens for a consented tenant on an ongoing basis, without the need for further tenant admin interaction. These access tokens are called app-only tokens because they do not include information about the tenant admin.

4) Call the Office 365 Management APIs. The app-only access tokens are passed to the Office 365 Management APIs to authenticate and authorize your application.

The following diagram shows the sequence of consent and access token requests.

Before you can access data through the Office 365 Management Activity API, you must enable unified audit logging for your Office 365 organization. You do this by turning on the Office 365 audit log. For instructions around enabling unified audit logging for your Office 365 organization kindly follow the link: https://docs.microsoft.com/en-us/microsoft-365/compliance/turn-audit-log-search-on-or-off?view=o365-worldwide

Note: Enabling unified audit logging isn't required if you're only using the Office 365 Service Communications API.

Register your application in Azure AD

The Office 365 Management APIs use Azure AD to provide secure authentication to Office 365 tenant data. To access the Office 365 Management APIs, you need to register your app in Azure AD, and as part of the configuration, you will specify the permission levels your app needs to access the APIs.

Utilize the Azure Management Portal to register your application in Azure AD

Post you have Microsoft tenant with required subscriptions in place, you can register your application in Azure AD.

1) Sign into the Azure management portal, using the credential of your Microsoft tenant that has the subscription to Office 365 you wish to use. You can also access the Azure Management Portal via a link that appears in the left navigation pane in the Office admin portal.

2) In the left navigation panel, choose Active Directory (1). Make sure the Directory tab (2) is selected, and then select the directory name (3).

3) On the directory page, select Applications. Azure AD displays a list of the applications currently installed in your tenancy.

4) Choose Add.

5) Select Add an application my organization is developing.

6) Enter the NAME of your application and specify the Type as WEB APPLICATION AND/OR WEB API.

7) Enter the appropriate App properties: o

• SIGN-ON URL. The URL where users can sign in and use your app. You can change this later as needed.

• APP ID URI. The URI used as a unique logical identifier for your app. The URI must be in a verified custom domain for an external user to grant your app access to their data in Windows Azure AD. For example, if your Microsoft tenant is contoso.onmicrosoft.com, the APP ID URI could be https://app.contoso.onmicrosoft.com.

8) Your app is now registered with Azure AD, and has been assigned a client ID. However, there are several important aspects of your app left to configure.

Configure your application properties in Azure AD

Now that your application is registered, there are several important properties you must specify that determine how your application functions within Azure AD and how tenant admins will grant consent to allow your application to access their data by using the Office 365 Management APIs.

For more information about Azure AD application configuration in general, kindly navigate the following link:

https://docs.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals

1) CLIENT ID. This value is automatically generated by Azure AD. Your application will use this value when requesting consent from tenant admins and when requesting app-only tokens from Azure AD.

2) APPLICATION IS MULTI-TENANT. This property must be set to YES to allow tenant admins to grant consent to your app to access their data by using the Office 365 Management APIs. If this property is set to NO, your application will only be able to access your own tenant's data.

3) REPLY URL. This is the URL that a tenant admin will be redirected to after granting consent to allow your application to access their data by using the Office 365 Management APIs. You can configure multiple reply URLs as needed. Azure automatically sets the first one to match the sign-on URL you specified when you created the application, but you can change this value as needed.

Please make sure to choose Save after making any changes to these properties.

Generate a new key for your application Keys, also known as client secrets, are used when exchanging an authorization code for an access token.

1) In the Azure Management Portal, select your application and choose Configure in the top menu. Scroll down to keys.

2) Select the duration for your key and choose Save.

3) Azure displays the app secret only after saving it. Select the Clipboard icon to copy the client secret to the Clipboard.

Important Note:

Azure only displays the client secret at the time you initially generate it. You cannot navigate back to this page and retrieve the client secret later.

Configure an X.509 certificate to enable service-to-service calls

An application that is running in the background, such as a daemon or service, can use client credentials to request app-only access tokens without repeatedly requesting consent from the tenant admin after initial consent is granted.

For more information kindly find the following link: https://docs.microsoft.com/en-us/previous-versions/azure/dn645543(v=azure.100)?redirectedfrom=MSDN

You must configure an X.509 certificate with your application to be used as client credentials when requesting app-only access tokens from Azure AD. There are two steps to the process:

1) Obtain an X.509 certificate. You can use a self-signed certificate, or a certificate issued by publicly trusted certificate authority.

2) Modify your application manifest to include the thumbprint and public key of your certificate.

The following instructions show you how to use the Visual Studio or Windows SDK makecert tool to generate a self-signed certificate and export the public key to a base64-encoded file.

From the command line, run the following:

makecert -r -pe -n "CN=MyCompanyName MyAppName Cert" -b 03/15/2015 -e 03/15/2017 -ss my -len 2048

Note: When you are generating the X.509 certificate, make sure the key length is at least 2048. Shorter key lengths are not accepted as valid keys.

1) Open the Certificates MMC snap-in and connect to your user account.

2) Find the new certificate in the Personal folder and export the public key to a base64-encoded file (for example, mycompanyname.cer). Your application will use this certificate to communicate with Azure AD, so make sure you retain access to the private key as well.

Note: You can use Windows PowerShell to extract the thumbprint and base64-encoded public key. Other platforms provide similar tools to retrieve properties of certificates.

3) From the Windows PowerShell prompt, type and run the following:

PowerShellCopy

$cer = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2

$cer.Import("mycer.cer")

$bin = $cer.GetRawCertData()

$base64Value = [System.Convert]::ToBase64String($bin)

$bin = $cer.GetCertHash()

$base64Thumbprint = [System.Convert]::ToBase64String($bin)

$keyid = [System.Guid]::NewGuid().ToString()

4) Store the values for $base64Thumbprint, $base64Value, and $keyid to be used when you update your application manifest in the next set of steps.

Using the values extracted from the certificate and the generated key ID, you must now update your application manifest in Azure AD.

5) In the Azure Management Portal, select your application and choose Configure in the top menu.

6) In the command bar, choose Manage manifest, and then choose Download Manifest.

7) Open the downloaded manifest for editing and replace the empty KeyCredentials property with the following JSON:

JSONCopy

"keyCredentials": [ { "customKeyIdentifier" : "$base64Thumbprint_from_above", "keyId": "$keyid_from_above", "type": "AsymmetricX509Cert", "usage": "Verify", "value": "$base64Value_from_above" } ],

"keyCredentials": [
{
"customKeyIdentifier" : "$base64Thumbprint_from_above",
"keyId": "$keyid_from_above",
"type": "AsymmetricX509Cert",
"usage": "Verify",
"value": "$base64Value_from_above"
}
],

Note: The KeyCredentials property is a collection, making it possible to upload multiple X.509 certificates for rollover scenarios or delete certificates for compromise scenarios.

8) Save your changes and upload the updated manifest by choosing Manage manifest in the command bar, choosing Upload manifest, browsing to your updated manifest file, and then selecting it.

Specify the permissions your app requires to access the Office 365 Management APIs

Finally, you need to specify exactly what permissions your app requires of the Office 365 Management APIs. To do so, you add access to the Office 365 Management APIs to your app, and then you specify the permission(s) you need. In the Azure Management Portal, select your application, and choose Configure in the top menu. Scroll down to permissions to other applications and choose Add application.

Select the Office 365 Management APIs (1) so that it appears in the Selected column (2), and then select the check mark in the lower right (3) to save your selection and return to the main configuration page for your application.

The Office Management APIs now appear in the list of applications to which your application requires permissions. Under both Application Permissions and Delegated Permissions, select the permissions your application requires. Refer to the specific API reference for more details about each permission.

Note: There are currently four unused permissions related to activity reports and threat intelligence that will be removed in the future. Do not select any of these permissions because they are unnecessary.

Choose Save to save the configuration.

Get Office 365 tenant admin consent

Now that your application is configured with the permissions it needs to use the Office 365 Management APIs, a tenant admin must explicitly grant your application these permissions in order to access their tenant's data by using the APIs. To grant consent, the tenant admin must sign into Azure AD by using the following specially constructed URL, where they can review your application's requested permissions. This step is not required when using the APIs to access data from your own tenant.

HTTP https://login.windows.net/common/oauth2/authorize?response_type=code&resource=https%3A%2F%2Fmanage.office.com&client_id={your_client_id}&redirect_uri={your_redirect_url}

The redirect URL must match or be a sub-path under one of the Reply URLs configured for your application in Azure AD.

For example:

HTTP: https://login.windows.net/common/oauth2/authorize?response_type=code&resource=https%3A%2F%2Fmanage.office.com&client_id=2d4d11a2-f814-46a7-890a-274a72a7309e&redirect_uri=http%3A%2F%2Fwww.mycompany.com%2Fmyapp%2F

You can test the consent URL by pasting it into a browser and signing in using the credentials of an Office 365 admin for a tenant other than the tenant that you used to register the application. You will see the request to grant your application permission to use the Office Management APIs.

After choosing Accept, you are redirected to the specified page, and there will be a code in the query string.

For example:

HTTP: http://www.mycompany.com/myapp/?code=AAABAAAAvPM1KaPlrEqdFSB...

Your application uses this authorization code to obtain an access token from Azure AD, from which the tenant ID can be extracted. After you have extracted and stored the tenant ID, you can obtain subsequent access tokens without requiring the tenant admin to sign in.

Request access tokens from Azure AD

There are two methods for requesting access tokens from Azure AD: The Authorization Code Grant Flow involves a tenant admin granting explicit consent, which returns an authorization code to your application. Your application then exchanges the authorization code for an access token. This method is required to obtain the initial consent that your application needs to access the tenant data by using the API, and this first access token is needed in order to obtain and store the tenant ID.

The Client Credentials Grant Flow allows your application to request subsequent access tokens as old ones expire, without requiring the tenant admin to sign in and explicitly grant consent. This method must be used for applications that run continuously in the background calling the APIs once the initial tenant admin consent has been granted.

Request an access token using the authorization code

After a tenant admin grants consent, your application receives an authorization code as a query string parameter when Azure AD redirects the tenant admin to your designated URL.

HTTP: http://www.mycompany.com/myapp/?code=AAABAAAAvPM1KaPlrEqdFSB...

Your application makes an HTTP REST POST to Azure AD to exchange the authorization code for an access token. Because the tenant ID is not yet known, the POST will be to the "common" endpoint, which does not have the tenant ID embedded in the URL:

HTTP: https://login.windows.net/common/oauth2/token

The body of the POST contains the following:

JSONCopy

resource=https%3A%2F%2Fmanage.office.com&client_id=a6099727-6b7b-482c-b509-1df309acc563 &redirect_uri= http%3A%2F%2Fwww.mycompany.com%2Fmyapp%2F &client_secret={your_client_key}&grant_type=authorization_code&code= AAABAAAAvPM1KaPlrEqdFSB...

Sample request

JSON

POST https://login.windows.net/common/oauth2/token HTTP/1.1

Content-Type: application/x-www-form-urlencoded

Host: login.windows.net

Content-Length: 944

resource=https%3A%2F%2Fmanage.office.com&client_id=a6099727-6b7b-482c-b509-1df309acc563 &redirect_uri= http%3A%2F%2Fwww.mycompany.com%2Fmyapp%2F &client_secret={your_client_key}&grant_type=authorization_code&code=AAABAAAAvPM1KaPlrEqdFSB...

The body of the response will include several properties, including the access token.

Sample response

JSON

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Content-Length: 3265

{"expires_in":"3599","token_type":"Bearer","scope":"ActivityFeed.Read ActivityReports.Read ServiceHealth.Read","expires_on":"1438290275","not_before":"1438286375","resource":"https://manage.office.com","access_token":"eyJ0eX...","refresh_token":"AAABAAA...","id_token":"eyJ0eXAi..."}

The access token that is returned is a JWT token that includes information about both the admin that granted consent and the application requesting access. The following shows an example of an un-encoded token. Your application must extract the tenant ID "tid" from this token and store it so that it can be used to request additional access tokens as they expire, without further admin interaction.

Sample token

JSON:

{ "aud": "https://manage.office.com", "iss": "https://sts.windows.net/41463f53-8812-40f4-890f-865bf6e35190/", "iat": 1427246416, "nbf": 1427246416, "exp": 1427250316, "ver": "1.0", "tid": "41463f53-8812-40f4-890f-865bf6e35190", "amr": [ "pwd" ], "oid": "1cef1fdb-ff52-48c4-8e4e-dfb5ea83d357", "upn": "[email protected]", "puid": "1003BFFD8EC47CA6", "sub": "7XpD5OWAXM1OWmKiVKh1FOkKXV4N3OSRol6mz1pxxhU", "given_name": "John", "family_name": "Doe", "name": "Contoso, Inc.", "unique_name": "[email protected]", "appid": "a6099727-6b7b-482c-b509-1df309acc563", "appidacr": "1", "scp": "ActivityFeed.Read ServiceHealth.Read", "acr": "1" }

{
"aud": "https://manage.office.com",
"iss": "https://sts.windows.net/41463f53-8812-40f4-890f-865bf6e35190/",
"iat": 1427246416,
"nbf": 1427246416,
"exp": 1427250316,
"ver": "1.0",
"tid": "41463f53-8812-40f4-890f-865bf6e35190",
"amr": [
"pwd"
],
"oid": "1cef1fdb-ff52-48c4-8e4e-dfb5ea83d357",
"upn": "[email protected]",
"puid": "1003BFFD8EC47CA6",
"sub": "7XpD5OWAXM1OWmKiVKh1FOkKXV4N3OSRol6mz1pxxhU",
"given_name": "John",
"family_name": "Doe",
"name": "Contoso, Inc.",
"unique_name": "[email protected]",
"appid": "a6099727-6b7b-482c-b509-1df309acc563",
"appidacr": "1",
"scp": "ActivityFeed.Read ServiceHealth.Read",
"acr": "1"
}

Request an access token by using client credentials

After the tenant ID is known, your application can make service-to-service calls to Azure AD to request additional access tokens as they expire. These tokens include information only about the requesting application and not about the admin that originally granted consent. Service-to-service calls require that your application use an X.509 certificate to create client assertion in the form of a base64-encoded, SHA256 signed JWT bearer token. When you are developing your application in .NET, you can use the Azure AD Authentication Library (ADAL) to create client assertions. Other development platforms should have similar libraries. An un-encoded JWT token consists of a header and payload that have the following properties.

JSON

HEADER:

{ "alg": "RS256", "x5t": "{thumbprint of your X.509 certificate used to sign the token", } PAYLOAD: { "aud": "https://login.windows.net/{tenantid}/oauth2/token", "iss": "{your app client ID}", "sub": "{your app client ID}" "jti": "{random GUID}", "nbf": {epoch time, before which the token is not valid}, "exp": {epoch time, after which the token is not valid}, }

{
"alg": "RS256",
"x5t": "{thumbprint of your X.509 certificate used to sign the token",
}
PAYLOAD:
{
"aud": "https://login.windows.net/{tenantid}/oauth2/token",
"iss": "{your app client ID}",
"sub": "{your app client ID}"
"jti": "{random GUID}",
"nbf": {epoch time, before which the token is not valid},
"exp": {epoch time, after which the token is not valid},
}

SAMPLE JWT TOKEN:

HEADER:

{ "alg": "RS256", "x5t": "YyfshJC3rPQ-kpGo5dUaiY5t3iU", } PAYLOAD: { "aud": "https://login.windows.net/41463f53-8812-40f4-890f-865bf6e35190/oauth2/token", "iss": "a6099727-6b7b-482c-b509-1df309acc563", "sub": "a6099727-6b7b-482c-b509-1df309acc563" "jti": "0ce254c4-81b1-4a2e-8436-9a8c3b49dfb9", "nbf": 1427248048, "exp": 1427248648, }

{
"alg": "RS256",
"x5t": "YyfshJC3rPQ-kpGo5dUaiY5t3iU",
}
PAYLOAD:
{
"aud": "https://login.windows.net/41463f53-8812-40f4-890f-865bf6e35190/oauth2/token",
"iss": "a6099727-6b7b-482c-b509-1df309acc563",
"sub": "a6099727-6b7b-482c-b509-1df309acc563"
"jti": "0ce254c4-81b1-4a2e-8436-9a8c3b49dfb9",
"nbf": 1427248048,
"exp": 1427248648,
}

The client assertion is then passed to Azure AD as part of a service-to-service call to request an access token. When using client credentials to request an access token, use an HTTP POST to a tenant-specific endpoint, where the previously extracted and stored tenant ID is embedded in the URL.

HTTP:

https://login.windows.net/{tenantid}/oauth2/token

The body of the POST contains the following:

JSONCopy

resource=https%3A%2F%2Fmanage.office.com&client_id={your_app_client_id}&grant_type=client_credentials&client_assertion_type=urn%3Aietf%3Aparams%3Aoauth%3Aclient-assertion-type%3Ajwt-bearer&client_assertion={encoded_signed_JWT_token}

Sample request:

JSON

POST https://login.windows.net/41463f53-8812-40f4-890f-865bf6e35190/oauth2/token HTTP/1.1

Content-Type: application/x-www-form-urlencoded

Host: login.windows.net

Content-Length: 994

resource=https%3A%2F%2Fmanage.office.com&client_id= a6099727-6b7b-482c-b509-1df309acc563&grant_type=client_credentials &client_assertion_type=urn%3Aietf%3Aparams%3Aoauth%3Aclient-assertion-type%3Ajwt-bearer&client_assertion=eyJhbGciOiJSUzI1NiIsIng1dCI6Ill5ZnNoSkMzclBRLWtwR281ZFVhaVk1dDNpVSJ9.eyJhdWQiOiJodHRwczpcL1wvbG9naW4ud2luZG93cy5uZXRcLzQxNDYzZjUzLTg4MTItNDBmNC04OTBmLTg2NWJmNmUzNTE5MFwvb2F1dGgyXC90b2tlbiIsImV4cCI6MTQyNzI0ODY0OCwiaXNzIjoiYTYwOTk3MjctNmI3Yi00ODJjLWI1MDktMWRmMzA5YWNjNTYzIiwianRpIjoiMGNlMjU0YzQtODFiMS00YTJlLTg0MzYtOWE4YzNiNDlkZmI5IiwibmJmIjoxNDI3MjQ4MDQ4LCJzdWIiOiJhNjA5OTcyNy02YjdiLTQ4MmMtYjUwOS0xZGYzMDlhY2M1NjMifQ.vfDrmCjiXgoj2JrTkwyOpr-NOeQTzlXQcGlKGNpLLe0oh4Zvjdcim5C7E0UbI3Z2yb9uKQdx9G7GeqS-gVc9kNV_XSSNP4wEQj3iYNKpf_JD2ikUVIWBkOg41BiTuknRJAYOMjiuBE2a6Wyk-vPCs_JMd7Sr-N3LiNZ-TjluuVzWHfok_HWz_wH8AzdoMF3S0HtrjNd9Ld5eI7MVMt4OTpRfh-Syofi7Ow0HN07nKT5FYeC_ThBpGiIoODnMQQtDA2tM7D3D6OlLQRgLfI8ir73PVXWL7V7Zj2RcOiooIeXx38dvuSwYreJYtdphmrDBZ2ehqtduzUZhaHL1iDvLlw

The response will be the same as before, but the token will not have the same properties, because it does not contain properties of the admin that granted consent.

Sample response

JSONCopy

HTTP/1.1 200 OK

Content-Type: application/json; charset=utf-8

Content-Length: 1276

{"token_type":"Bearer","expires_in":"3599","expires_on":"1431659094","not_before":"1431655194","resource":"https://manage.office.com","access_token":"eyJ0eXAiOiJKV1QiL..."} Sample access token JSONCopy { "aud": "https://manage.office.com", "iss": "https://sts.windows.net/41463f53-8812-40f4-890f-865bf6e35190/", "iat": 1431655194, "nbf": 1431655194, "exp": 1431659094, "ver": "1.0", "tid": "41463f53-8812-40f4-890f-865bf6e35190", "roles": [ "ServiceHealth.Read", "ActivityFeed.Read" ], "oid": "67cb0334-e242-4783-8028-0f39132fb5ad", "sub": "67cb0334-e242-4783-8028-0f39132fb5ad", "idp": "https://sts.windows.net/41463f53-8812-40f4-890f-865bf6e35190/", "appid": "a6099727-6b7b-482c-b509-1df309acc563", "appidacr": "1" }

{
"token_type":"Bearer","expires_in":"3599","expires_on":"1431659094","not_before":"1431655194","resource":"https://manage.office.com","access_token":"eyJ0eXAiOiJKV1QiL..."}
Sample access token
JSONCopy
{
"aud": "https://manage.office.com",
"iss": "https://sts.windows.net/41463f53-8812-40f4-890f-865bf6e35190/",
"iat": 1431655194,
"nbf": 1431655194,
"exp": 1431659094,
"ver": "1.0",
"tid": "41463f53-8812-40f4-890f-865bf6e35190",
"roles": [
"ServiceHealth.Read",
"ActivityFeed.Read"
],
"oid": "67cb0334-e242-4783-8028-0f39132fb5ad",
"sub": "67cb0334-e242-4783-8028-0f39132fb5ad",
"idp": "https://sts.windows.net/41463f53-8812-40f4-890f-865bf6e35190/",
"appid": "a6099727-6b7b-482c-b509-1df309acc563",
"appidacr": "1"
}

Build your app

Now that you have registered your app in Azure AD and configured it with the necessary permissions, you are ready to build your app. The following are some of the key aspects to consider when designing and building your app:

The consent experiences to obtain consent from your customers, you must direct them in a browser to the Azure AD website, using the specially constructed URL described previously, and you must have a website to which Azure AD will redirect the admin once they grant consent. This website must extract the authorization code from the URL and use it to request an access token from which it can obtain the tenant ID.

Store the tenant ID in your system. This will be needed when requesting access tokens from Azure AD and when calling the Office Management APIs.

Managing access tokens. You will need a component that requests and manages access tokens as needed. If your app calls the APIs periodically, it can request tokens on demand, or if it calls the APIs continuously to retrieve data, it can request tokens at regular intervals (for example, every 45 minutes).

Implement a webhook listener as needed by the particular API you are using.

Data retrieval and storage. You will need a component that retrieves data for each tenant, either by using continuous polling or in response to webhook notifications, depending on the particular API you are using.