[SOLVED] Importing a protobuf file from another directory and building it with CMake

Issue

I have a project directory structure with protobuf messages defined in different directories. Basically, msg_b.proto imports msg_a.proto so that it can use message A as a field. The directory structure is like,

proto
|-- CMakeLists.txt
`-- src
    |-- CMakeLists.txt
    |-- msg_a
    |   |-- CMakeLists.txt
    |   `-- msg_a.proto
    `-- msg_b
        |-- CMakeLists.txt
        `-- msg_b.proto

3 directories, 6 files

the top level CMakeLists.txt is define like,

project(proto)
cmake_minimum_required(VERSION 3.18)

find_package(Protobuf REQUIRED)

add_subdirectory(src)

the CMakeLists.txt file under proto/src is like,

add_subdirectory(msg_a)
add_subdirectory(msg_b)

and the CMakeLists.txt files for msg_a and msg_b are like,

protobuf_generate_cpp(PROTO_HDRS PROTO_SRCS msg_a.proto)

add_library(msg_a ${PROTO_HDRS} ${PROTO_SRCS})
target_link_libraries(msg_a protobuf::libprotobuf)

and

set(Protobuf_IMPORT_DIRS ${CMAKE_SOURCE_DIR})
protobuf_generate_cpp(PROTO_HDRS PROTO_SRCS msg_b.proto)

add_library(msg_b ${PROTO_HDRS} ${PROTO_SRCS})
target_include_directories(msg_b PUBLIC ${CMAKE_BINARY_DIR})
target_link_libraries(msg_b protobuf::libprotobuf)

respectively. The actual proto files are just,

syntax = "proto2";

package dummy.package;

message A {
    optional int32 a = 1;
    optional int32 b = 2;
    optional double c = 3;
}

and

syntax = "proto2";

package dummy.package.msb_b;

import "src/msg_a/msg_a.proto";

message B {
    optional dummy.package.A a = 1;
}

When I try and build this though, I get the following error with both gcc and clang,

[build] [ 50%] Building CXX object src/msg_a/CMakeFiles/msg_a.dir/msg_a.pb.cc.o
[build] [ 66%] Building CXX object src/msg_b/CMakeFiles/msg_b.dir/msg_b.pb.cc.o
[build] /home/aobrien/dev/cpp/proto/build/src/msg_b/msg_b.pb.cc:17:8: error: use of undeclared identifier 'PROTOBUF_INTERNAL_EXPORT_src_2fmsg_5fa_2fmsg_5fa_2eproto'
[build] extern PROTOBUF_INTERNAL_EXPORT_src_2fmsg_5fa_2fmsg_5fa_2eproto ::PROTOBUF_NAMESPACE_ID::internal::SCCInfo<0> scc_info_A_src_2fmsg_5fa_2fmsg_5fa_2eproto;
[build]        ^
[build] /home/aobrien/dev/cpp/proto/build/src/msg_b/msg_b.pb.cc:70:6: error: no member named 'descriptor_table_src_2fmsg_5fa_2fmsg_5fa_2eproto' in the global namespace; did you mean 'descriptor_table_protodef_msg_5fb_2eproto'?
[build]   &::descriptor_table_src_2fmsg_5fa_2fmsg_5fa_2eproto,
[build]    ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[build]      descriptor_table_protodef_msg_5fb_2eproto
[build] /home/aobrien/dev/cpp/proto/build/src/msg_b/msg_b.pb.cc:64:12: note: 'descriptor_table_protodef_msg_5fb_2eproto' declared here
[build] const char descriptor_table_protodef_msg_5fb_2eproto[] PROTOBUF_SECTION_VARIABLE(protodesc_cold) =
[build]            ^
[build] /home/aobrien/dev/cpp/proto/build/src/msg_b/msg_b.pb.cc:70:3: error: cannot initialize an array element of type 'const ::google::protobuf::internal::DescriptorTable *const' with an rvalue of type 'const char (*)[92]'
[build]   &::descriptor_table_src_2fmsg_5fa_2fmsg_5fa_2eproto,
[build]   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[build] 3 errors generated.

I've been trying to figure this out for hours without luck :(. Where am I going wrong?

I've been googling around and haven't found much. If I put both of the messages in the same directory, then I can build it successfully but that would be a big pain for my project. I basically want to use CMake to build my protobufs, but not have dependencies across directories.

Solution

Protobuf encodes the relative path of the file that is being passed to the generator (protobuf_generate_cpp) in the names of the identifiers. So when you generate the first file in msg_a, then those identifiers will not have any relative path. But when you now include the file in msg_b, this will clash, which is why you see the identifier mismatch.

Relative directories and protobuf don't mix very well, especially if they aren't generated by the same protobuf generation step, so don't use a relative path with directories in the include, but pretend the files are next to each other:

msg_b's CMakeLists.txt:

set(Protobuf_IMPORT_DIRS ${CMAKE_CURRENT_SOURCE_DIR}/../msg_a)
protobuf_generate_cpp(PROTO_HDRS PROTO_SRCS msg_b.proto)

add_library(msg_b ${PROTO_HDRS} ${PROTO_SRCS})
target_include_directories(msg_b PUBLIC ${CMAKE_CURRENT_BINARY_DIR}/../msg_a)
target_link_libraries(msg_b PUBLIC protobuf::libprotobuf)
target_link_libraries(msg_b PUBLIC msg_a)

msg_b.proto:

syntax = "proto2";

package dummy.package.msb_b;

import "msg_a.proto";

message B {
    optional dummy.package.A a = 1;
}

Answered By - chris_se

Answer Checked By - Katrina (WPSolving Volunteer)