schema

Delta Lake: Schema Enforcement & Evolution

Reading Time: 4 minutes Nowadays data is constantly evolving and changing. As well as the business problems and requirements are evolving, the shape or the structure of the data is also changing. When that happens, we want to be in control of how the data or schema changes. But how we can achieve this? Delta Lake has good ways to control how schema changes. With Delta Lake, users have Continue Reading

Error Registering Avro Schema | Multiple Schemas In One Topic

Reading Time: 4 minutes org.apache.kafka.common.errors.SerializationException: Error registering Avro schema: {“type”:”record”,”name”:”schema1″,”namespace”:”test”,”fields”:[{“name”:”Name”,”type”:”string”},{“name”:”Age”,”type”:”int”},{“name”:”Location”,”type”:”string”}]} Caused by: io.confluent.kafka.schemaregistry.client.rest.exceptions.RestClientException: Schema being registered is incompatible with an earlier schema; error code: 409 at io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:170) at io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:188) at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:245) at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:237) at io.confluent.kafka.schemaregistry.client.rest.RestService.registerSchema(RestService.java:232) at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.registerAndGetId(CachedSchemaRegistryClient.java:59) at io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.register(CachedSchemaRegistryClient.java:91) at io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl(AbstractKafkaAvroSerializer.java:72) at io.confluent.kafka.formatter.AvroMessageReader.readMessage(AvroMessageReader.java:158) at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:57) at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala) You might have come across a similar exception while working with AVRO schemas. Kafka throws this exception due to a compatibility issue Continue Reading