You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We are trying to apply the Envelope to a JSON structured log and the output does not conform to the data model expected by the OTEL standard.
To Reproduce
Example log message:
{"message":"The entertaining kiwi reveals itself as a plucky giraffe to those who look. Before raspberries, monkeys were only apricots. A pro-active zebra without rabbits is truly a apple of calm chimpanzees. A grape is an orange's lemon!","timestamp_app":"2024-09-06T19:35:25.767494843Z","log_type":"APPLICATION","log_level":"ALERT","event":{"data":"98In7cO5Fc"},"http_request":"GET /shouting-with-happiness-one-cannot-separate-deers-from-inventive-tigers","http_method":"GET","X-Request-ID":"bc5a0dec-df87-48ae-a049-a55d928ee0f0","custom_fields":{"custom.field_1":"vw","custom.field_2":"g","custom.field_3":"c0LwCH","custom.field_4":"Sg6f0W","service.name":"poc-opensearch-logs"},"trace_id":"9e44b233-9b2f-4656-a4e5-994d1da6ea18","span_id":"8a9a4217-0808-4ef6-91e4-a9f2eb4435ee","trace_flags":"0102040800000000"}
{"message":"The entertaining kiwi reveals itself as a plucky giraffe to those who look. Before raspberries, monkeys were only apricots. A pro-active zebra without rabbits is truly a apple of calm chimpanzees. A grape is an orange's lemon!","timestamp_app":"2024-09-06T19:35:25.767494843Z","log_type":"APPLICATION","log_level":"ALERT","event":{"data":"98In7cO5Fc"},"http_request":"GET /shouting-with-happiness-one-cannot-separate-deers-from-inventive-tigers","http_method":"GET","X-Request-ID":"bc5a0dec-df87-48ae-a049-a55d928ee0f0","custom_fields":{"custom.field_1":"vw","custom.field_2":"g","custom.field_3":"c0LwCH","custom.field_4":"Sg6f0W","service.name":"poc-opensearch-logs"},"trace_id":"9e44b233-9b2f-4656-a4e5-994d1da6ea18","span_id":"8a9a4217-0808-4ef6-91e4-a9f2eb4435ee","trace_flags":"0102040800000000"}
{"message":"The entertaining kiwi reveals itself as a plucky giraffe to those who look. Before raspberries, monkeys were only apricots. A pro-active zebra without rabbits is truly a apple of calm chimpanzees. A grape is an orange's lemon!","timestamp_app":"2024-09-06T19:35:25.767494843Z","log_type":"APPLICATION","log_level":"ALERT","event":{"data":"98In7cO5Fc"},"http_request":"GET /shouting-with-happiness-one-cannot-separate-deers-from-inventive-tigers","http_method":"GET","X-Request-ID":"bc5a0dec-df87-48ae-a049-a55d928ee0f0","custom_fields":{"custom.field_1":"vw","custom.field_2":"g","custom.field_3":"c0LwCH","custom.field_4":"Sg6f0W","service.name":"poc-opensearch-logs"},"trace_id":"9e44b233-9b2f-4656-a4e5-994d1da6ea18","span_id":"8a9a4217-0808-4ef6-91e4-a9f2eb4435ee","trace_flags":"0102040800000000"}
{"message":"The entertaining kiwi reveals itself as a plucky giraffe to those who look. Before raspberries, monkeys were only apricots. A pro-active zebra without rabbits is truly a apple of calm chimpanzees. A grape is an orange's lemon!","timestamp_app":"2024-09-06T19:35:25.767494843Z","log_type":"APPLICATION","log_level":"ALERT","event":{"data":"98In7cO5Fc"},"http_request":"GET /shouting-with-happiness-one-cannot-separate-deers-from-inventive-tigers","http_method":"GET","X-Request-ID":"bc5a0dec-df87-48ae-a049-a55d928ee0f0","custom_fields":{"custom.field_1":"vw","custom.field_2":"g","custom.field_3":"c0LwCH","custom.field_4":"Sg6f0W","service.name":"poc-opensearch-logs"},"trace_id":"9e44b233-9b2f-4656-a4e5-994d1da6ea18","span_id":"8a9a4217-0808-4ef6-91e4-a9f2eb4435ee","trace_flags":"0102040800000000"}
{"message":"The entertaining kiwi reveals itself as a plucky giraffe to those who look. Before raspberries, monkeys were only apricots. A pro-active zebra without rabbits is truly a apple of calm chimpanzees. A grape is an orange's lemon!","timestamp_app":"2024-09-06T19:35:25.767494843Z","log_type":"APPLICATION","log_level":"ALERT","event":{"data":"98In7cO5Fc"},"http_request":"GET /shouting-with-happiness-one-cannot-separate-deers-from-inventive-tigers","http_method":"GET","X-Request-ID":"bc5a0dec-df87-48ae-a049-a55d928ee0f0","custom_fields":{"custom.field_1":"vw","custom.field_2":"g","custom.field_3":"c0LwCH","custom.field_4":"Sg6f0W","service.name":"poc-opensearch-logs"},"trace_id":"9e44b233-9b2f-4656-a4e5-994d1da6ea18","span_id":"8a9a4217-0808-4ef6-91e4-a9f2eb4435ee","trace_flags":"0102040800000000"}
{"message":"The entertaining kiwi reveals itself as a plucky giraffe to those who look. Before raspberries, monkeys were only apricots. A pro-active zebra without rabbits is truly a apple of calm chimpanzees. A grape is an orange's lemon!","timestamp_app":"2024-09-06T19:35:25.767494843Z","log_type":"APPLICATION","log_level":"ALERT","event":{"data":"98In7cO5Fc"},"http_request":"GET /shouting-with-happiness-one-cannot-separate-deers-from-inventive-tigers","http_method":"GET","X-Request-ID":"bc5a0dec-df87-48ae-a049-a55d928ee0f0","custom_fields":{"custom.field_1":"vw","custom.field_2":"g","custom.field_3":"c0LwCH","custom.field_4":"Sg6f0W","service.name":"poc-opensearch-logs"},"trace_id":"9e44b233-9b2f-4656-a4e5-994d1da6ea18","span_id":"8a9a4217-0808-4ef6-91e4-a9f2eb4435ee","trace_flags":"0102040800000000"}
{"message":"The entertaining kiwi reveals itself as a plucky giraffe to those who look. Before raspberries, monkeys were only apricots. A pro-active zebra without rabbits is truly a apple of calm chimpanzees. A grape is an orange's lemon!","timestamp_app":"2024-09-06T19:35:25.767494843Z","log_type":"APPLICATION","log_level":"ALERT","event":{"data":"98In7cO5Fc"},"http_request":"GET /shouting-with-happiness-one-cannot-separate-deers-from-inventive-tigers","http_method":"GET","X-Request-ID":"bc5a0dec-df87-48ae-a049-a55d928ee0f0","custom_fields":{"custom.field_1":"vw","custom.field_2":"g","custom.field_3":"c0LwCH","custom.field_4":"Sg6f0W","service.name":"poc-opensearch-logs"},"trace_id":"9e44b233-9b2f-4656-a4e5-994d1da6ea18","span_id":"8a9a4217-0808-4ef6-91e4-a9f2eb4435ee","trace_flags":"0102040800000000"}
Using the following configuration file (fluentbit):
{
"resourceLogs": [
{
"resource": {
"attributes": [
{
"key": "cloud.availability_zone",
"value": {
"stringValue": "zone-1"
}
}
]
},
"scopeLogs": [
{
"scope": {},
"logRecords": [
{
"timeUnixNano": "1725651721220525000",
"body": {
"stringValue": "{\"log\"=>\"{\"message\":\"The entertaining kiwi reveals itself as a plucky giraffe to those who look. Before raspberries, monkeys were only apricots. A pro-active zebra without rabbits is truly a apple of calm chimpanzees. A grape is an orange's lemon!\",\"timestamp_app\":\"2024-09-06T19:35:25.767494843Z\",\"log_type\":\"APPLICATION\",\"log_level\":\"ALERT\",\"event\":{\"data\":\"98In7cO5Fc\"},\"http_request\":\"GET /shouting-with-happiness-one-cannot-separate-deers-from-inventive-tigers\",\"http_method\":\"GET\",\"X-Request-ID\":\"bc5a0dec-df87-48ae-a049-a55d928ee0f0\",\"custom_fields\":{\"custom.field_1\":\"vw\",\"custom.field_2\":\"g\",\"custom.field_3\":\"c0LwCH\",\"custom.field_4\":\"Sg6f0W\",\"service.name\":\"poc-opensearch-logs\"},\"trace_id\":\"9e44b233-9b2f-4656-a4e5-994d1da6e"
},
"traceId": "",
"spanId": ""
}
]
}
]
}
]
}
Your Environment
Version used: 3.1.7
Configuration: attached
Environment name and version (e.g. Kubernetes? What version?): k8s
The text was updated successfully, but these errors were encountered:
I tested it with unstructured log and it worked perfectly but unfortunately not with the structured example I posted. I even tried to change the lib of one of my applications to generate in OTEL format but then I had another challenge with the OpenTelemetry Output #9372.
I tested it with unstructured log and it worked perfectly but unfortunately not with the structured example I posted. I even tried to change the lib of one of my applications to generate in OTEL format but then I had another challenge with the OpenTelemetry Output #9372.
I have fixed it by a very strange way, use filebeat to collect my logs,use logstash to receive them and output to the specific files,then insert into mysql with fluentd.
Bug Report
We are trying to apply the Envelope to a JSON structured log and the output does not conform to the data model expected by the OTEL standard.
To Reproduce
Your Environment
The text was updated successfully, but these errors were encountered: