Secret spelled out on Scarbble board

Using app secrets in #dotnetcore console applications

I was writing a sample dotnetcore console application for a talk because why I felt using a sample aspnet core web app was overkill. The app was connecting to a bunch of Azure cloud and 3rd party services (think Twilio API for SMS or LaunchDarkly API for Feature Flags) and I had to deal with connection strings.

Now I have a nasty habit of "accidentally" checking in connection string and secrets into public GitHub repositories, so I wanted to do this right from the get go.

I started with this official documentation on adding configuration in a new .NET console application. To start with, add a package reference to Microsoft.Extensions.Hosting (example with dotnet cli)

https://gist.github.com/desinole/5a734a136616986582aa3126b8dfb8a6.js

Next, modify Program.cs to instantiate a new instance of the HostBuilder class with pre-configured defaults

https://gist.github.com/desinole/c83622a2214d83d6ed118352c675e576.js

According to the Microsoft docs, The Host.CreateDefaultBuilder(String[]) method provides default configuration for the app in the following order:

  1. ChainedConfigurationProvider : Adds an existing IConfiguration as a source.
  2. appsettings.json using the JSON configuration provider.
  3. appsettings.Environment.json using the JSON configuration provider. For example, appsettings.Production.json and appsettings.Development.json.
  4. App secrets when the app runs in the Development environment.
  5. Environment variables using the Environment Variables configuration provider.
  6. Command-line arguments using the Command-line configuration provider.

While options 1, 2, 3, 5 and 6 sound like they would work, I would really like to use option 4 since it explicitly talks about app secrets and I believe secrets should be stored separate from config. Additionally, I really liked the app secrets experience, including support for POCO, when I worked with aspnetcore web applications. So, I decided to go with option 4.

But the Microsoft docs takes you a page that shows you Safe storage of app secrets in development in ASP.NET Core. This might work but I would like to keep my application as a simple console app not a web app. Still there are some instructions that would be useful on here.

The app secret values will be stored in a JSON file in the local machine’s user profile folder. For Windows this path will be %APPDATA%\Microsoft\UserSecrets\secrets.json and for Linux or MacOS this path will be ~/.microsoft/usersecrets//secrets.json.

Use the Secrets Manager command line to enable secrets for the project

https://gist.github.com/desinole/aae97d5faea8752e83b05b45a4c1e15e.js

This puts a guid in your .csproj file. If you take this guid and plug it into the paths above instead of , you will get access to the the secrets.json file that will store the secrets for the application, in case you want to review them during debugging. Please note, the init function will not create the file. The file will be created and modified as you continue to add and modify secrets.

Next, we come up with POCO classes that can be used to map to the secrets. In this case, I split MyAppSecrets into AzureSecrets and ExternalSecrets and have dedicated sub-classes for each.

https://gist.github.com/desinole/909dabfe8d82dd8c84fdd8b8445158d4.js

Now to add the secrets themselves. The dotnet user secrets tool does not store nested properties as proper json. Instead it uses a : to separate the properties structure. Let’s take the above POCO classes, to access SQLConnectionString, we would use TopLevelClassName:PropertyClassName:PropertyName as the secret name, for instance, MyAppSecrets:CloudSecrets:SQLConnectionString. To set a secret, you would use a dotnet cli command as below

https://gist.github.com/desinole/07d485552b239e7575dc0b9a1f93c48a.js

Repeating this for all the sub-classes and properties you could end up with a secrets.json file that looks something like below – not very readable, I know

https://gist.github.com/desinole/5b013c952c9abecd5ee2ec484292756f.js

Now that we have the secrets stored and the POCO class to retrieve it, app secrets should be enabled when we run the app in development environment. It turns out you can tell the HostBuilder to use a particular environment right after instantiating it.

Host.CreateDefaultBuilder(args).UseEnvironment("development");

Convenient, isn’t it? You can add and delete the UseEnvironment("development") code snippet and witness for yourself how Secrets are added and removed from the list of configuration sources by debug watching the sources variable in below code.

        static IHostBuilder CreateHostBuilder(string[] args) =>
            Host.CreateDefaultBuilder(args).UseEnvironment("development")
            .ConfigureAppConfiguration((hostingContext, configuration) => {
                var sources = configuration.Sources;
            });

But I really don’t want to have to deal with all these source since I’m only interested in the App Secrets source. I can do this using a configuration.Sources.Clear() call from the CreateHostBuilder method. Having done that I can build a configuration source for my app secrets and read it on to a static variable to use in the program as follow

Add a static variable for the Program class

static IConfiguration Configuration;

Then modify your HostBuilder method to build an app secrets configuration.

        static IHostBuilder CreateHostBuilder(string[] args) =>
            Host.CreateDefaultBuilder(args).UseEnvironment("development")
            .ConfigureAppConfiguration((hostingContext, configuration) => {
                configuration.Sources.Clear();
                Configuration = configuration.AddUserSecrets<MyAppSecrets>().Build(); 
            });

And boom goes the dynamite. Now you can use the static variable in your Main class to either access your secrets as a key value pair or by mapping it to a strongly-typed class as shown below

static async Task Main(string[] args)
{
    using IHost host = CreateHostBuilder(args).Build();
    //mapping static variable to strongly typed class
    var appSecrets = Configuration.GetSection(nameof(MyAppSecrets)).Get<MyAppSecrets>();
    Console.WriteLine($"Strongly typed mapping {appSecrets.CloudSecrets.SQLConnectionString}");
    //access secrets using key value pair
    Console.WriteLine($"Key value pair {Configuration["MyAppSecrets:CloudSecrets:SQLConnectionString"]}");
    await host.RunAsync();
}

But wait! There’s more. If you don’t want to have to deal with using the dotnetcore cli to enter awkwardly structured secrets like MyAppSecrets:CloudSecrets:SQLConnectionString, you can modify the secrets.json file directly with properly formatted JSON value and your strongly type mapping will ensure that these values are available to you either as strongly typed class properties or config key value pairs

{
	"MyAppSecrets":{
	  "UtilitySecrets":{
		  "LaunchDarklyApiKey": "launchdarklyconnstring",
		  "TwilioApiKey": "twilioconnstring"
	  },
	  "CloudSecrets":{
		  "SQLConnectionString": "sqlconnstring",
		  "CosmosConnectionString": "cosmosdbconnstring"
	  }
	}
}

Feel free to peruse the easy to read source code. I will add more instructions in the GitHub repo but this blog should suffice for now.

4 thoughts on “Using app secrets in #dotnetcore console applications”

  1. This is a great piece and makes me realize you have gone far beyond the hack I created for myself when I encountered the same in the docs. I wonder if we can propose making a solution an official documented solution. Many times I have been writing small console apps that will be tiny servers and they have secrets too.

    1. Thanks for the feedback, Christopher. Let me see if I can get some code reviews to see if this is acceptable as an official solution. Might be useful to others also

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.