Page MenuHomePhabricator

Research avro schema evolution, do we need a write and reader schema? [3]
Closed, ResolvedPublic

Description

Research avro schema evolution, do we need a write and reader schema?

Turns out that yes, we do.

Event Timeline

Nuria raised the priority of this task from to Needs Triage.
Nuria updated the task description. (Show Details)
Nuria added a project: Analytics-Kanban.
Nuria subscribed.

See encoding of the same data with schema1 and schema1 with an additional "optional" field. Note that the absence of the "additional " field is represented by an empty byte.

[nuriaruiz@nurieta][/workplace/analytics/refinery/source/refinery-camus/src/test/resources/avro]$ more ./hexdump.original
00000000 a4 13 08 74 6f 74 6f 08 62 6c 61 68 0a 31 32 33 |...toto.blah.123|
00000010 34 35 06 31 32 33 0c 31 32 33 34 35 36 06 0a 74 |45.123.123456..t|
00000020 65 73 74 31 0a 74 65 73 74 32 0a 74 65 73 74 33 |est1.test2.test3|
00000030 00 00 |..|
00000032
[nuriaruiz@nurieta][/workplace/analytics/refinery/source/refinery-camus/src/test/resources/avro]$ more ./hexdump.new
00000000 a4 13 08 74 6f 74 6f 08 62 6c 61 68 0a 31 32 33 |...toto.blah.123|
00000010 34 35 06 31 32 33 0c 31 32 33 34 35 36 06 0a 74 |45.123.123456..t|
00000020 65 73 74 31 0a 74 65 73 74 32 0a 74 65 73 74 33 |est1.test2.test3|
00000030 00 00 00 |...|
00000033

Nuria renamed this task from Research avro schema evolution, do we need a write and reader schema? to Research avro schema evolution, do we need a write and reader schema? [3].Nov 20 2015, 5:03 PM
Nuria set Security to None.
Nuria moved this task from In Progress to Done on the Analytics-Kanban board.