Wednesday, November 3, 2021

[SOLVED] Unable to transfer data from MS Sql Server to ElasticSearch through Logstash

Issue

I am trying to synchronize MS Sql Server data with ElasticSearch through Logstash I am setting up the configuration as suggested in Elastic documentation. All the things(MS SQL,ES and LogStash) are setup on the same server which has the environment of CentOS 7

I have tried to setup every possible JDBC Jar with it which were compatible with my JDK(1.8.0_131). Moreover I have provided the correct path to the Jar also.

Below is my SQL.conf:

input {
  jdbc {
    jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
    jdbc_driver_library => "/root/sqljdbc_6.4/enu/mssql-jdbc-6.4.0.jre8.jar"
    jdbc_connection_string => "jdbc:sqlserver://localhost:1433;databaseName=test"
     jdbc_user => "sa"
     jdbc_password => "abc.123"
    statement => "select * from Service"

  }
}

output {
  elasticsearch {
    hosts => ["http://localhost:9200/"]
    index => "Service"
        document_id => "%{service_id}"
  }
}

Error: com.microsoft.sqlserver.jdbc.SQLServerDriver not loaded. Are you sure you've included the correct jdbc driver in :jdbc_driver_library?


Solution

I suggest moving the JAR file to somewhere else than under /root as Logstash might not be able to read there:

jdbc_driver_library => "/tmp/sqljdbc_6.4/enu/mssql-jdbc-6.4.0.jre8.jar"
                          ^
                          |
                   change the folder


Answered By - Val