Don't use PretrainedModelInitializer when loading a model

[eladsegal]

When loading a model, pretrained embeddings are not loaded as it is not needed.
This change make the same thing happen with pretrained weights from PretrainedModelInitializer.

Goes with allenai/allennlp-models#141

[dirkgr]

This seems fine, but wouldn't it be easier to just not run initializers at all when loading the model?

[eladsegal]

This seems fine, but wouldn't it be easier to just not run initializers at all when loading the model?

If we make the assumption that for every module the initializer is under "initializer" in the config, then yes.
Should I make the change?

[dirkgr]

I went hunting for a more elegant way of doing this than grepping for the word "initializer" in a Params object. But then I found the remove_pretrained_embedding_params() function that already does a similar thing exactly that way. Can you just adapt that function, or write something like it for initializers?

[dirkgr]

Fantastic, thank you!