有个问题，我设置了分区为-1，但是kafka看消息好像都在一个分区呢 #1775

Activity

1. kedixa commented on Sep 11, 2025

   

   kedixa

   on Sep 11, 2025

   Contributor

   More actions

   使用的哪个版本的代码，从源码上看不应该出现这种情况，在源码的这个1512行的地方加个日志，打印一下partition和partition_num看看有没有运行到这里吧

   workflow/src/client/WFKafkaClient.cc

   Lines 1498 to 1514 in 0f1bbce

   	if (partition_num <= 0)
   	return -1;
   	int partition = -1;
   	if (this->partitioner)
   	{
   	const void *key;
   	size_t key_len;
   	record->get_key(&key, &key_len);
   	partition = this->partitioner(toppar->get_topic(), key,
   	key_len, partition_num);
   	}
   	else
   	partition = rand() % partition_num;
   	KafkaToppar *new_toppar = get_toppar(toppar->get_topic(),
   	partition,
2. hotwinters commented on Sep 12, 2025

   

   hotwinters

   on Sep 12, 2025

   Author

   More actions

   @kedixa 我无法自己调用接口获取元数据吗，我尝试在初始化后时候取了下，但发现meta为空
   bool KafkaProducer::init(const std::string &brokers)
   {
   if (initialized_)
   {
   spdlog::get("app_logger")->error("KafkaProducer already initialized");
   return false;
   }
   brokers_ = brokers;
   if (client_.init(brokers_) != 0)
   {
   spdlog::get("app_logger")->error("Kafka client init failed, brokers: {}", brokers.c_str());
   return false;
   }
   initialized_ = true;

   const KafkaMeta *meta;
   meta = get_meta("frame_topic_1",client_.get_meta_list());
   if (!meta)
   	return false;
   
   auto partition_num = meta->get_partition_elements();
   if (partition_num <= 0)
       return fasle;
   
   return true;
   

   }
3. kedixa commented on Sep 12, 2025

   

   kedixa

   on Sep 12, 2025

   Contributor

   More actions

   init不会立刻取meta，可以在produce成功后的callback中通过这种方式获取meta信息
4. hotwinters commented on Sep 12, 2025

   

   hotwinters

   on Sep 12, 2025

   Author

   More actions

   @kedixa 感谢大佬回复，我添加到callback中监控了，然后发现是在运行过程中运维添加了新的分区，但这时候我getmeta获取的还是缓存的数据，有没有办法能跟随运维动态更改meta呢
5. kedixa commented on Sep 12, 2025

   

   kedixa

   on Sep 12, 2025

   Contributor

   More actions

   目前WFKafkaClient没有定时更新topic meta的功能，只会在发现错误时触发更新，而新增分区不会触发错误。
6. Barenboim commented on Sep 17, 2025

   

   Barenboim

   on Sep 17, 2025

   Contributor

   More actions

   @hotwinters 这个问题现在你那边能解决么？或者有没有任何优化的建议？