Agfdhyk

I have simple fluentd conf:

<source>

  @type tail

  @id in_tail_springboot_container_logs

  #path /var/log/containers/*.log

  path /var/log/logtest.txt

  #path /var/log/containers/springbootrabbitmqlistener-*.log

  pos_file /var/log/fluentd-springboot-containers.log.pos

  @log_level debug

  multiline_flush_interval 5s

  format multiline

  format1 (?<timestamp>d{4}-d{2}-d{2} d{2}:d{2}:d{2}.d{3})  (?<level>INFO|ERROR|WARN|TRACE|DEBUG|FATAL)s+[(?<app>[^,]+),(?<zipkintraceid>[^,]*),(?<zipkinspanid>[^,]*),(?<zipkinsent>[^]]*)] (?<someid>[0-9]+) --- [(?<thread>[^]]+)] (?<classname>[^ ]+)s+: (?<text>.*)

  format_firstline ?<timestamp>d{4}-d{2}-d{2} d{2}:d{2}:d{2}.d{3}

  tag springboot.*

  read_from_head true

  <parse>

    @type json

    time_format %Y-%m-%dT%H:%M:%S.%NZ

  </parse>

</source>

I test it with input line:

2018-11-09 08:49:36.111  INFO [SpringBootMicroservice,,,] 5820 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Bean with name 'environmentManager' has been autodetected for JMX exposure

I've tested it with https://regex101.com/.

Everything works fine.

Problem is that fluent doesn't like it:

2018-11-09 08:48:18 +0000 [warn]: #0 [in_tail_springboot_container_logs] pattern not match: "2018-11-09 08:49:36.111  INFO [SpringBootMicroservice,id1,id2,true] 5820 --- [           main] o.s.j.e.a.AnnotationMBeanExporter        : Bean with name 'environmentManager' has been autodetected for JMX exposure"

Can you tell me what is wrong with my conf?

It looks like the problem is with this line:

format_firstline ?<timestamp>d{4}-d{2}-d{2} d{2}:d{2}:d{2}.d{3}

The issue is that the timestamp named group is incorrectly formatted - It needs parenthesis around the entire group, like so:

                 ↓                                                      ↓

format_firstline (?<timestamp>d{4}-d{2}-d{2} d{2}:d{2}:d{2}.d{3})

                                                                 ↑

And for good practice, (as the Regex veteran Wiktor mentioned) you should escape your .'s when they are meant to be literal dot's, not wildcards.