Core: Move Hadoop conf serialization into SerializableConfiguration

[nastra]

This moves the Hadoop-specific config serialization out of SerializableTable into Hadoop's SerializableConfiguration

[rdblue]

Should we deprecate this constructor and the serializeConfWith method? That method will double-wrap the configuration if it is called. I like that this should now be Kryo-compatible as well.

[nastra]

it looks like at least the method is being used in

iceberg/mr/src/main/java/org/apache/iceberg/mr/mapreduce/IcebergInputFormat.java

Lines 175 to 183 in a2802c4

	private void checkAndSkipIoConfigSerialization(Configuration config, Table table) {
	if (table != null
	&& config.getBoolean(
	InputFormatConfig.CONFIG_SERIALIZATION_DISABLED,
	InputFormatConfig.CONFIG_SERIALIZATION_DISABLED_DEFAULT)
	&& table.io() instanceof HadoopConfigurable) {
	((HadoopConfigurable) table.io())
	.serializeConfWith(conf -> new NonSerializingConfig(config)::get);
	}

[RussellSpitzer]

I think there is a question of whether that is important enough to keep this method. As far as I can tell it's just for an optimization of not serializing the Hadoop Conf?

It's only opt in (CONFIG_SERIALIZATION_DISABLED) is false by default and I'm dubious it's even that beneficial now. Since we are going to be serializing this all over the place in non-hadoop situations right? I would recommend we just deprecate it then remove it.

[nastra]

ok I've deprecated both

[RussellSpitzer]

Two other places that may need updating?

SerializationUtil.serializeToBytes line 46:

iceberg/core/src/main/java/org/apache/iceberg/util/SerializationUtil.java

Line 46 in df2dc69

return serializeToBytes(obj, conf -> new SerializableConfiguration(conf)::get);

ResolvingFileIO.setConf line 157:

iceberg/core/src/main/java/org/apache/iceberg/io/ResolvingFileIO.java

Line 157 in df2dc69

this.hadoopConf = new SerializableConfiguration(conf)::get;

[nastra]

thanks everyone for the reviews, I'll go ahead and merge this