[zoom] Add support for the meeting activity data stream

packages/zoom/_dev/build/docs/README.md

@@ -7,18 +7,19 @@
 This integration collects data using two complementary methods:
 
 - **Webhook**: a real-time HTTP listener that receives event notifications pushed by Zoom (meeting, webinar, recording, user, account, phone, team chat, and Zoom Room events).
-- **REST API**: a periodic poll of the Zoom REST API to collect the sign in / sign out **activity** report and the **operation** logs report for an account.
+- **REST API**: a periodic poll of the Zoom REST API to collect the sign in / sign out **activity** report, the **operation** logs report, and the **meeting_activity** logs report for an account.
 
 ### Compatibility
 
 - The **activity** data stream uses the Zoom REST API [`GET /report/activities`](https://developers.zoom.us/docs/api/meetings/#tag/reports/get/report/activities) endpoint and requires a Zoom Pro (or higher) plan.
 - The **operation** data stream uses the Zoom REST API [`GET /report/operationlogs`](https://developers.zoom.us/docs/api/meetings/#tag/reports/get/report/operationlogs) endpoint and requires a Zoom Pro plan or above.
+- The **meeting_activity** data stream uses the Zoom REST API [`GET /report/meeting_activities`](https://developers.zoom.us/docs/api/meetings/#tag/reports/get/report/meeting_activities) endpoint. The meeting audit trail log feature must be enabled for the account by Zoom Support.
 
 ### How it works
 
 The **webhook** data stream creates an HTTP listener that accepts incoming webhook callbacks from Zoom. The Elastic Agent running this integration must be reachable from the internet so that Zoom can connect to it. Zoom requires that webhooks are delivered over HTTPS, so you must either configure the integration with a valid TLS certificate or place a reverse proxy that terminates TLS in front of the integration. Incoming events are then routed to the appropriate ingest pipeline based on the Zoom event type.
 
-The **activity** and **operation** data streams both poll the Zoom REST API using Server-to-Server OAuth. On each interval, they request records within a date window (a maximum of one month per request, within the last six months of available history) and paginate through the results.
+The **activity**, **operation** and **meeting_activity** data streams poll the Zoom REST API using Server-to-Server OAuth. On each interval, they request records within a date window (a maximum of one month per request) and paginate through the results.
 
 ## What data does this integration collect?
 
@@ -27,6 +28,7 @@
 - `webhook`: real-time Zoom event notifications, including account, team chat (channel and message), meeting, phone, recording, user, webinar, and Zoom Room events.
 - `activity`: account-wide sign in and sign out activity logs from the Zoom REST API reports endpoint. Note that the API does not provide data for failed sign-in or authentication attempts, so those logs will not be available here.
 - `operation`: account-wide admin and user operation logs from the Zoom REST API reports endpoint, such as adding a user, changing account settings, or deleting a recording.
+- `meeting_activity`: meeting activity logs from the Zoom REST API reports endpoint, such as a meeting being created or started, a user joining or leaving, in-meeting chat, remote control, and a meeting ending.
 
 ### Supported use cases
 
@@ -35,6 +37,7 @@
 - **Webhook** events enable real-time monitoring across meetings, recordings, users, and account changes.
 - The **activity** report provides an account-wide sign in / sign out audit trail for investigating user access and anomalous logins.
 - The **operation** logs report tracks admin and user operations for auditing configuration changes and detecting unauthorized actions.
+- The **meeting_activity** logs report provides a meeting-level audit trail of meeting lifecycle and participant activity — meetings being created, started, and ended, participants joining and leaving, in-meeting chat, and remote control — helping you reconstruct what happened in a specific meeting, monitor meeting usage, and support compliance investigations.
 
 ## What do I need to use this integration?
 
@@ -50,7 +53,7 @@
 
 1. Create a **Server-to-Server OAuth** app in the [Zoom App Marketplace](https://marketplace.zoom.us/) by following the [Server-to-Server OAuth documentation](https://developers.zoom.us/docs/internal-apps/s2s-oauth/).
 2. Record the app's **Account ID**, **Client ID**, and **Client Secret**.
-3. Add the `report:read:admin` scope (or the granular `report:read:user_activities:admin` & `report:read:operation_logs:admin` scopes) to the app and activate it. A Zoom Pro plan or above is required.
+3. Add the `report:read:admin` scope (or the granular `report:read:user_activities:admin`, `report:read:operation_logs:admin` & `report:read:meeting_activity_log:admin` scopes) to the app and activate it. A Zoom Pro plan or above is required.
 
 ## How do I deploy this integration?
 
@@ -95,7 +98,7 @@
 
 ### Zoom REST API rate limits
 
-The REST API data streams (`activity` and `operation`) query Zoom's Report endpoints, which are classified as **Heavy** APIs. Zoom enforces both a per-second (QPS) limit and a daily request quota, and **both are shared across every app and user on the account, as well as across all of the Report data streams in this integration**:
+The REST API data streams (`activity`, `operation`, and `meeting_activity`) query Zoom's Report endpoints, which are classified as **Heavy** APIs. Zoom enforces both a per-second (QPS) limit and a daily request quota, and **both are shared across every app and user on the account, as well as across all of the Report data streams in this integration**:
 
 | Plan | Per second | Per day (shared by Heavy and Resource-intensive APIs) |
 |---|---|---|
@@ -141,6 +144,14 @@
 
 {{fields "operation"}}
 
+### meeting_activity
+
+This is the `meeting_activity` data stream. It collects meeting activity logs from the Zoom REST API.
+
+{{event "meeting_activity"}}
+
+{{fields "meeting_activity"}}
+
 ### Inputs used
 
 These inputs are used in this integration:
@@ -154,3 +165,4 @@
 
 - `activity`: [Get sign in / sign out activity report](https://developers.zoom.us/docs/api/meetings/#tag/reports/get/report/activities).
 - `operation`: [Get operation logs report](https://developers.zoom.us/docs/api/meetings/#tag/reports/get/report/operationlogs).
+- `meeting_activity`: [Get a meeting activities report](https://developers.zoom.us/docs/api/meetings/#tag/reports/get/report/meeting_activities).

packages/zoom/_dev/deploy/docker/docker-compose.yml

@@ -10,6 +10,19 @@ services:
       - STREAM_ADDR=http://elastic-agent:9080/zoom
       - STREAM_WEBHOOK_HEADER=Authorization=abc123
     command: log --start-signal=SIGHUP --delay=5s /sample_logs/account-ndjson.log
+  zoom-meeting-activity:
+    image: docker.elastic.co/observability/stream:v0.20.0
+    hostname: zoom-meeting-activity
+    ports:
+      - 8090
+    volumes:
+      - ./files:/files:ro
+    environment:
+      PORT: '8090'
+    command:
+      - http-server
+      - --addr=:8090
+      - --config=/files/config-meeting_activity.yml
   zoom-operation:
     image: docker.elastic.co/observability/stream:v0.20.0
     hostname: zoom-operation

packages/zoom/_dev/deploy/docker/files/config-meeting_activity.yml

@@ -0,0 +1,134 @@
+rules:
+  # Zoom S2S OAuth token endpoint (account_credentials grant).
+  - path: /oauth/token
+    methods: [POST]
+    query_params:
+      grant_type: account_credentials
+      account_id: test-account-id
+    request_headers:
+      Authorization:
+        - "Basic dGVzdC1jbGllbnQtaWQ6dGVzdC1jbGllbnQtc2VjcmV0"
+      Content-Type:
+        - "application/x-www-form-urlencoded"
+    responses:
+      - status_code: 200
+        headers:
+          Content-Type:
+            - "application/json"
+        body: |
+          {"access_token":"test-access-token","token_type":"Bearer","expires_in":3600,"scope":"report:read:admin"}
+  # Pagination page 2 — token from page 1.
+  - path: /v2/report/meeting_activities
+    methods: [GET]
+    query_params:
+      from: "{from:.*}"
+      to: "{to:.*}"
+      activity_type: "{activity_type:.*}"
+      page_size: "{page_size:.*}"
+      next_page_token: mock-page-2-token
+    responses:
+      - status_code: 200
+        headers:
+          Content-Type:
+            - "application/json"
+        body: |-
+          {
+            "meeting_activity_logs": [
+              {
+                "meeting_number": "982 610 0285",
+                "activity_time": "{{ .request.vars.from }} 07:35:12:880",
+                "operator": "Bob Brown",
+                "operator_email": "bob@example.com",
+                "activity_category": "User left",
+                "activity_detail": "Bob Brown left the meeting"
+              }
+            ],
+            "page_size": 300,
+            "next_page_token": "mock-page-3-token"
+          }
+  # Pagination page 3 — terminal page for round 1.
+  - path: /v2/report/meeting_activities
+    methods: [GET]
+    query_params:
+      from: "{from:.*}"
+      to: "{to:.*}"
+      activity_type: "{activity_type:.*}"
+      page_size: "{page_size:.*}"
+      next_page_token: mock-page-3-token
+    responses:
+      - status_code: 200
+        headers:
+          Content-Type:
+            - "application/json"
+        body: |-
+          {
+            "meeting_activity_logs": [
+              {
+                "meeting_number": "982 610 0285",
+                "activity_time": "{{ .request.vars.from }} 07:40:00:120",
+                "operator": "Jill Chill",
+                "operator_email": "jillchill@example.com",
+                "activity_category": "Meeting ended",
+                "activity_detail": "Meeting ended"
+              }
+            ],
+            "page_size": 300,
+            "next_page_token": ""
+          }
+  # Initial page (round 1) and cursor-resumed page (round 2).
+  - path: /v2/report/meeting_activities
+    methods: [GET]
+    query_params:
+      from: "{from:.*}"
+      to: "{to:.*}"
+      activity_type: "{activity_type:.*}"
+      page_size: "{page_size:.*}"
+    responses:
+      - status_code: 200
+        headers:
+          Content-Type:
+            - "application/json"
+        body: |-
+          {{ if eq .req_num 1 }}
+          {
+            "meeting_activity_logs": [
+              {
+                "meeting_number": "982 610 0285",
+                "activity_time": "{{ .request.vars.from }} 07:09:03:216",
+                "operator": "Jill Chill",
+                "operator_email": "jillchill@example.com",
+                "activity_category": "Meeting Started",
+                "activity_detail": "Meeting Started"
+              },
+              {
+                "meeting_number": "982 610 0285",
+                "activity_time": "{{ .request.vars.from }} 07:09:45:002",
+                "operator": "Bob Brown",
+                "operator_email": "bob@example.com",
+                "activity_category": "User joined",
+                "activity_detail": "Bob Brown joined the meeting"
+              }
+            ],
+            "page_size": 300,
+            "next_page_token": "mock-page-2-token"
+          }
+          {{ else if eq .req_num 2 }}
+          {
+            "meeting_activity_logs": [
+              {
+                "meeting_number": "112 233 4455",
+                "activity_time": "{{ .request.vars.from }} 09:00:10:500",
+                "operator": "Alice Adams",
+                "operator_email": "alice@example.com",
+                "activity_category": "Meeting created",
+                "activity_detail": "Meeting created"
+              }
+            ],
+            "page_size": 300
+          }
+          {{ else }}
+          {
+            "meeting_activity_logs": [],
+            "page_size": 300
+          }
+          {{ end }}

packages/zoom/changelog.yml

@@ -1,4 +1,9 @@
 # newer versions go on top
+- version: "1.26.0"
+  changes:
+    - description: Add support for the `meeting_activity` data stream.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/19554
 - version: "1.25.0"
   changes:
     - description: Add support for the `operation` data stream.

packages/zoom/data_stream/meeting_activity/_dev/test/pipeline/test-common-config.yml

@@ -0,0 +1,3 @@
+fields:
+  tags:
+    - preserve_original_event

packages/zoom/data_stream/meeting_activity/_dev/test/pipeline/test-zoom-meeting-activity.log

@@ -0,0 +1,5 @@
+{"meeting_number":"982 610 0285","activity_time":"2024-03-21 07:09:03:216","operator":"Jill Chill","operator_email":"jillchill@example.com","activity_category":"Meeting Started","activity_detail":"Meeting Started"}
+{"meeting_number":"982 610 0285","activity_time":"2024-03-21 07:09:45:002","operator":"Bob Brown","operator_email":"bob@example.com","activity_category":"User joined","activity_detail":"Bob Brown joined the meeting"}
+{"meeting_number":"982 610 0285","activity_time":"2024-03-21 07:35:12:880","operator":"Bob Brown","operator_email":"bob@example.com","activity_category":"User left","activity_detail":"Bob Brown left the meeting"}
+{"meeting_number":"982 610 0285","activity_time":"2024-03-21 07:40:00:120","operator":"Jill Chill","operator_email":"jillchill@example.com","activity_category":"Meeting ended","activity_detail":"Meeting ended"}
+{"meeting_number":"112 233 4455","activity_time":"2024-03-22 09:00:10:500","operator":"Alice Adams","activity_category":"Meeting created","activity_detail":"Meeting created"}