NoMethodError while registering SDE files in ArcGIS Server 10.7.1

[jesa-esrich]

Environment:
Windows Server 2016
ArcGIS Server 10.7.1
Chef Client 14.14.29
Microsoft SQL Server 2017
Microsoft SQL Server Driver 17
ArcGIS Cookbook 3.4.0

While installing and configuring ArcGIS Server 10.7.1 with Chef the script runs into an error while registering SDE Files in ArcGIS Server (registering folders works):

• arcgis_enterprise_datasources[Create folder ] action create_folder (skipped due to only_if)

• arcgis_enterprise_datasources[Share folder ] action share_folder (skipped due to only_if)

• arcgis_enterprise_datasources[Register folder in ArcGIS Server] action register_folder

  • execute[Register folder Rasterdaten as datasource] action run
    • execute python C:\Chef\misc_scripts/RegisterFolderAsDatasource.py C:\Chef\misc_scripts/AdminConnection.ags Rasterdaten \[machinename]\szh.loc\geo\gdi\rasterdaten

• arcgis_enterprise_datasources[Register SDE files from folder as data source within ArcGIS Server] action register_sde_files_from_folder

  ================================================================================
  Error executing action register_sde_files_from_folder on resource 'arcgis_enterprise_datasources[Register SDE files from folder as data source within ArcGIS Server]'

  NoMethodError

  undefined method `each' for nil:NilClass

  Resource Declaration:

  In c:/chef/local-mode-cache/cache/cookbooks/arcgis-enterprise/recipes/datasources.rb

  105: arcgis_enterprise_datasources 'Register SDE files from folder as data source within ArcGIS Server' do
  106: not_if { node['arcgis']['datasources']['sde_files']['folder'].nil? }
  107: action :register_sde_files_from_folder
  108: end
  109:

  Compiled Resource:

  Declared in c:/chef/local-mode-cache/cache/cookbooks/arcgis-enterprise/recipes/datasources.rb:105:in `from_file'

  arcgis_enterprise_datasources("Register SDE files from folder as data source within ArcGIS Server") do
  action [:register_sde_files_from_folder]
  default_guard_interpreter :default
  declared_type :arcgis_enterprise_datasources
  cookbook_name "arcgis-enterprise"
  recipe_name "datasources"
  not_if { #code block }
  end

  System Info:

  chef_version=14.14.29
  platform=windows
  platform_version=10.0.14393
  ruby=ruby 2.5.7p206 (2019-10-01 revision 67816) [x64-mingw32]
  program_name=C:/opscode/chef/bin/chef-solo
  executable=C:/opscode/chef/bin/chef-solo

This worked perfectly fine for the ArcGIS Cookbook 3.3.1 and ArcGIS Server 10.6.1. The mentioned recipe and depending files (providers...etc.) also haven't changed between those version.

Does anyone know the cause for this error and how to resolve it?

[cameronkroeker]

@jesa-esrich

Have you also tried with the latest cookbook release v3.5.0?

https://github.com/Esri/arcgis-cookbook/releases/tag/v3.5.0

[jesa-esrich]

@cameronkroeker I will give it a try. However, the specified recipe "datasources" hasn't changed for 2 years. So I'm not quite sure what is causing the problem.
We will also try to register other sde files.

[jesa-esrich]

The problem persist with the newest cookbook release. And registering another SDE connection file also didn't help.

[cameronkroeker]

@jesa-esrich Would you be able to share your role json file? (Please scrub out any sensitive information such as passwords, hostnames, etc).

[jesa-esrich]

@cameronkroeker

Here's the role json. As a side note there are also some custom recipes (marked as custom in the run list):

{
"arcgis": {
"run_as_user": "domain\user",
"run_as_password": "PASSWORD",
"version": "10.7.1",
"repository": {
"archives": "\\machinenameShare\hbd\dev\@software\ESRI\ArcGIS Enterprise 10.7.1",
"setups": "E:\AutomatedDeployment\ArcGIS",
"patches": "E:\AutomatedDeployment\ArcGIS\Patches"
},
"iis": {
"keystore_file": "C:\chef\keystore\Certificate.pfx",
"keystore_password": "PASSWORD"
},
"web_adaptor": {
"admin_access": true
},
"server": {
"admin_username": "agsadmin",
"admin_password": "PASSWORD",
"install_system_requirements": false,
"security": {
"user_store_config": {
"type": "ASP_NET",
"class": "AGSMembershipProvider.AGSADMembershipProvider",
"properties": {
"adminUser": "domain\user",
"adminUserPassword": "PASSWORD"
}
},
"role_store_config": {
"type": "ASP_NET",
"class": "AGSMembershipProvider.AGSADRoleProvider",
"properties": {
"adminUser": "domain\user",
"adminUserPassword": "PASSWORD"
}
},
"privileges": {
"PUBLISH": ["domain\user2"],
"ADMINISTER": ["domain\user"]
}
},
"wa_name": "arcgis",
"install_dir": "E:\Programs\ArcGIS\Server",
"local_directories_root": "E:\arcgisserver",
"authorization_file": "E:\AutomatedDeployment\ArcGIS\Authorization_Files\Server.ecp",
"keystore_file": "C:\chef\keystore\Certificate.pfx",
"keystore_password": "PASSWORD",
"soc_max_heap_size":256
},
"python": {
"install_dir": "E:\Programs\Python"
},
"datasources": {
"block_data_copy": true,
"folders": [
{
"server_path": "E:\Mapfiles",
"create_folder": true,
"security_permissions": {
"authorize_arcgis_service_account": true,
"full_control_members": ["domain\user2"]
},
"share_folder": true,
"share_name": "Mapfiles",
"sharing_permissions": {
"full_control_members": ["domain\user2", "domain\user"]
},
"publish_folder": {
"identifier": "Mapfiles",
"publish_with_hostname": true
}
},{
"server_path": "E:\Data",
"create_folder": true,
"security_permissions": {
"authorize_arcgis_service_account": true,
"full_control_members": ["domain\user2", "domain\user"]
},
"share_folder": true,
"share_name": "Data",
"sharing_permissions": {
"full_control_members": ["domain\user2", "domain\user"]
},
"publish_folder": {
"identifier": "Data",
"publish_with_hostname": true
}
},{
"server_path": "E:\Data",
"publish_folder": {
"identifier": "Data_With_Alias",
"publish_alternative_path": "\\machinename\Data"
}
},{
"server_path": "E:\Mapfiles",
"publish_folder": {
"identifier": "Mapfiles_With_Alias",
"publish_alternative_path": "\\machinename\Mapfiles"
}
},{
"publish_folder": {
"identifier": "Rasterdaten",
"publish_alternative_path": "\\machinenameShare\geo\gdi\rasterdaten"
}
}
],
"sde_files": {
"folder": "\\machinenameShare\documentation_installation\arcgis_db_connectionfiles\ChangeFolder"
}
},
"backups": {
"site_backup_dir": "\\machinenameShare\Backups\ChangeFolder\ArcGISServerSite"
},
"misc": {
"script_directory": "E:\AutomatedDeployment\Scripts"
}
},
"run_list": [
"recipe[arcgis-enterprise::system]",
"recipe[esri-iis]",
"recipe[arcgis-enterprise::server]",
"recipe[custom-arcgis-enterprise::sql_server_client]",
"recipe[arcgis-enterprise::server_security]",
"recipe[arcgis-enterprise::server_wa]",
"recipe[arcgis-enterprise::datasources]",
"recipe[custom-arcgis-enterprise::backups]",
"recipe[arcgis-enterprise::patches]",
"recipe[custom-arcgis-enterprise::pagefile]"
]
}

[jesa-esrich]

@cameronkroeker We tested a couple of other things:

• Registering the SDE Connection files using an array: "files":["\\AnotherFolder\With\SDE\Files\File1.sde", "\\AnotherFolder\With\SDE\Files\File2.sde"] --> did also not work and we received the same error message

• our certificate seems to be somewhat wrong as the CN is not using the FQDN (only the machine name). Will check with our IT that we get the correct certificate. CHEF seems to use the CN for accessing the server website and registering the datasources (registering folders at least).

• However, the SDE connection files can be uploaded manually: either accessing the ArcGIS Server Manager using FQDN --> trusted certifcate or just using the machine name --> untrusted certificate

[cameronkroeker]

@jesa-esrich

Thank you for providing the role json file used. I suspect the error undefined method each for nil:NilClass is because the ['node']['arcgis']['datasources']['sde_files']['files'] attribute is missing from the role json file.

I was able to reproduce the same error. Try adding the following to the json file:

"sde_files": {
"folder": \\machinenameShare\documentation_installation\arcgis_db_connectionfiles\ChangeFolder,
“files”:[]
}

This workaround worked for me.

And to your point about: "CHEF seems to use the CN for accessing the server website and registering the datasources (registering folders at least)."

I suspect this is because of the following:

arcgis-cookbook/cookbooks/arcgis-enterprise/providers/datasources.rb

Line 204 in 50cbc0f

def get_server_url

Looks like if ['arcgis']['datasources']['server_config_url'] is not specified then it will construct the following url:

  `server_url = 'https://' + cert_cn + ":6443/arcgis/admin"`

So perhaps try defining ['arcgis']['datasources']['server_config_url'] attribute in the role json file as well.

[jesa-esrich]

@cameronkroeker
Thanks a lot for the workaround! Worked like a charm.
As a side note: this also has to be done if the sde file connection files are registered using the files parameter. Hence in this case the folder parameter has to be passed as empty value "folder": ""

--> would be great to adjust/mention this in the documentation. Quite weirdly this was not needed for ArcGIS Cookbook 3.3.1 and ArcGIS Server 10.6.1. Maybe there was some dependency with the chef client.

Concerning the suggestion for the certificate, the server url parameter did not resolve it. It still uses the CN for accessing the server and registering the datasources. But I only ran the scripts isolated using the datasources recipe. I'll keep you up to date if something changes if I run the whole list of recipes.
The workaround would be to use the FQDN as CN in the certificate.

[cameronkroeker]

@jesa-esrich

Glad to hear that the workaround resolved the issue.

--> would be great to adjust/mention this in the documentation. Quite weirdly this was not needed for ArcGIS Cookbook 3.3.1 and ArcGIS Server 10.6.1. Maybe there was some dependency with the chef client.

This very well could be the case as there hasn't been any changes made to the datasources recipe in quite some time. I've made an internal issue for us to adjust the logic as this isn't the desired behavior with the "folders":"" and "files":"" attributes. We are also going to evaluate the server_config_url logic as well as this looks like it can be improved upon.

Thanks for bringing this to our attention and apologies that it took so long to find a workaround.

[cameronkroeker]

This has been fixed in cookbook release v3.6.1:

https://github.com/Esri/arcgis-cookbook/releases