trend_micro_vision_one: Support UNIX_MS date format in detection data stream - #14939
Merged
Conversation
kcreddy
commented
Aug 14, 2025
| {"eventTimeDT":"2022-07-11T13:35:34+00:00","tags":null,"uuid":"1234abcd-1234abcd-1234abcd-1234abcd-1234abcd","searchDL":"DDL","eventTime":1657546534000,"productCode":"sig","eventName":"WEB_POLICY_VIOLATION","rt_utc":"2022-07-11T13:35:34Z","rt":"2022-07-11T13:35:34+0000","act":["Block"],"aggregatedCount":"1","detectionType":"Web Reputation Service","deviceGUID":"456xyz-456xyz-456xyz-456xyz-456xyz","dst":["81.2.69.142"],"endpointGUID":"789pqr-789pqr-789pqr-789pqr-789pqr","endpointHostName":"ABC01","fileSize":"0","fileType":"ASCII Text","osName":"10.15.6","pname":"Secure Web Gateway","policyName":"default","pver":"1.0","request":"http://example.com/","requestBase":"example.com","src":["81.2.69.142"],"suid":"user.name","urlCat":["Newly Observed Domain"],"userDomain":"example.com","profile":"Default threat protection profile","principalName":"user.name@example.com","policyUuid":"123123-abcd-123-abcd","sender":"Default cloud gateway","logKey":"123-123-123-abc-abc-abc","clientIp":["81.2.69.142"]} | ||
| {"eventTimeDT":"2022-07-11T13:35:34+00:00","tags":null,"uuid":"1234abcd-1234abcd-1234abcd-1234abcd-1234abcd","searchDL":"DDL","eventTime":1657546534000,"productCode":"sig","eventName":"WEB_POLICY_VIOLATION","rt_utc":"2022-07-11T13:35:34Z","rt":"2022-07-11T13:35:34+0000","act":["Block"],"aggregatedCount":"1","detectionType":"Web Reputation Service","deviceGUID":"456xyz-456xyz-456xyz-456xyz-456xyz","dst":["81.2.69.142"],"endpointGUID":"789pqr-789pqr-789pqr-789pqr-789pqr","endpointHostName":"ABC01","fileSize":"0","fileType":"ASCII Text","osName":"10.15.6","pname":"Secure Web Gateway","policyName":"default","pver":"1.0","requests":"http://example.com/","requestBase":"example.com","src":["81.2.69.142"],"suid":"user.name","urlCat":["Newly Observed Domain"],"userDomain":"example.com","profile":"Default threat protection profile","principalName":"user.name@example.com","policyUuid":"123123-abcd-123-abcd","sender":"Default cloud gateway","logKey":"123-123-123-abc-abc-abc","clientIp":["81.2.69.142"]} | ||
| {"act":["not blocked"],"aggregatedCount":"1","app":"SMB2","appGroup":"CIFS","aptRelated":"0","clientFlag":"src","cnt":"8","compressedFileSize":"0","dceHash1":"0","dceHash2":"0","detectionType":"1","deviceDirection":"outbound","deviceGUID":"73487B369ACB-4179A816-9CE9-24B1-C575","deviceMacAddress":"00:11:22:33:44:55","devicePayloadId":"99:10786306:::","deviceRiskConfidenceLevel":3,"dhost":"samplehost","dmac":"00:00:0b:06:bc:87","dpt":445,"dst":["81.2.69.142"],"dstGroup":"Default","dstZone":"1","dvc":["81.2.69.142"],"dvchost":"EUABCDATE09","eventId":"100119","eventName":"SECURITY_RISK_DETECTION","eventSourceType":3,"eventTime":1696921180000,"eventTimeDT":"2023-10-10T06:59:40+00:00","filePath":" ","filePathName":" ","fileSize":"0","hasdtasres":"No","interestedGroup":"Default","interestedHost":"xxx-docker","interestedIp":["192.168.47.102"],"isHidden":"Yes","logKey":"123-123-123-abc-abc-abc","malType":"OTHERS","malTypeGroup":"Others","overSsl":"Not over SSL/TLS","pComp":"NCIE","peerGroup":"Default","peerHost":"samplehost","peerIp":["81.2.69.144"],"pname":"Deep Discovery Inspector","potentialRisk":"1","productCode":"pdi","pver":"6.5.1129","remarks":"[IP address: 81.2.69.144]","rt":"2023-10-10T06:59:40.0000000Z","rtDate":"2023-10-10T00:00:00.0000000Z","rtHour":10,"rtWeekDay":"Tuesday","rt_utc":"2023-10-10T06:59:40.0000000Z","ruleId":3498,"ruleName":"DOWNLOAD FILE - SMB2(REQUEST) - BETA","searchDL":"DDL","senderGUID":"73598B359BBF-4189A816-9AD9-24B1-C475","senderIp":["81.2.69.144"],"severity":2,"shost":"sampleshost","smac":"00:00:0b:06:bc:87","spt":52066,"src":["81.2.69.144"],"srcGroup":"Default","srcZone":"1","threatType":"99","uuid":"f4c08e8d-715e-4ac9-9469-2644b7cdt959","vLANId":4085} | ||
| {"attachment":[{"attachmentFileHash":"5264bc98832d51837a1dd3705da7fe01d8f065e7","attachmentFileName":"image.png","attachmentFileSize":"-1","attachmentFileTlsh":""}],"attachmentFileHashes":["5124ba98812d51837a1dd3705da7fd00d8f067e6"],"attachmentFileHashs":["5124ba98812d51837a1dd3705da7fd00d8f067e6"],"attachmentFileName":["image.png"],"attachmentFileSizes":["-1"],"attachmentFileTlshes":[""],"attachmentFileTlshs":[""],"dataType":1,"description":"The subject (\"SOLICITAÇÃO DE CORTES SEMANA 33\") of this email is similar to known malicious mail subjects.","duser":["user@example.com.br","user2@example.com.br"],"eventId":"100009","eventName":"MESSAGE_SUSPICIOUS_DETECTION","eventSourceType":2,"eventTime":1754591294000,"eventTimeDT":"2025-08-07T18:28:14+00:00","fileCreation":"","filterRiskLevel":"info","groupId":"ec333310-1111-11e8-bb99-af11bd8888c7","groupIdCorrKey":"SUBJECT","groupIdCorrValues":["SOLICITAÇÃO DE CORTES SEMANA 33"],"logReceivedTime":"1754591305681","mailDeliveryTime":"","mailMsgDirection":21,"mailMsgSubject":"SOLICITAÇÃO DE CORTES SEMANA 33","mailReceivedTime":"","mailbox":"j.user3@example.com.br","msgId":"\u003cPPPPP80BB706167454555533336BEAA51AAAAA@AAAAAAA.aaaaaa.prod.outlook.com\u003e","msgUuid":"BBkAFgAAAAAABBBDEapmEc2byACqZY-FHg0A89ecNkvrvjJBwcWvipKPygABpXdddgAA","objectFileCreation":"","objectFileModified":"","orgId":"ec333310-1111-11e8-bb99-af11bd8888c7","pname":"Email Sensor","productCode":"xms","rt":"1754591294000","rt_utc":"2025-08-07T18:28:25.2070000Z","ruleName":"MA-01-030","ruleVer":"","samUser":"j.user3","scanTs":"","scanType":"realtime_mailmeta-exchange","searchDL":"DDL","subRuleName":"malicious_data_cluster","suser":["j.user3@example.com.br"],"tags":["XSJG.MA-01-030","MITRE.T1566.002"],"uuid":"d7777944-10c9-4197-9d97-0f8e66ef6666"} |
Contributor
Author
There was a problem hiding this comment.
Taken from user sample where rt field is in UNIX_MS format.
kcreddy
added
enhancement
kcreddy
marked this pull request as ready for review
August 14, 2025 17:06
|
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
kcreddy
changed the title
Contributor
🚀 Benchmarks reportTo see the full report comment with |
💚 Build Succeeded
cc @kcreddy |
|
efd6
approved these changes
Aug 15, 2025
Contributor
|
Package trend_micro_vision_one - 2.3.0 containing this change is available at https://epr.elastic.co/package/trend_micro_vision_one/2.3.0/ |
tehbooom
pushed a commit
to tehbooom/integrations
that referenced
this pull request
Nov 19, 2025
… stream (elastic#14939) trend_micro_vision_one: Support more date formats in detection data stream. "detection" pipeline could receive date fields in UNIX_MS format, which is currently not supported. Add UNIX_MS date format option for all the date fields.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed commit message
Checklist
changelog.ymlfile.How to test this PR locally
Run pipeline tests (with new sample data containing
rtfield as UNIX_MS):Before:
After: