From a154ed003909d1308471f38cbe6f9bcc8f5379f8 Mon Sep 17 00:00:00 2001 From: Jon Sequeira Date: Tue, 11 Jul 2023 14:42:08 -0700 Subject: [PATCH] some miscellaneous docs improvements --- DEVELOPER-GUIDE.md | 68 +++++---- docs/formatting.md | 175 ++++++++++++++++++---- docs/javascript-overview.md | 8 +- src/polyglot-notebooks-vscode/DEVGUIDE.md | 43 ++++-- src/polyglot-notebooks-vscode/README.md | 24 +-- 5 files changed, 231 insertions(+), 87 deletions(-) diff --git a/DEVELOPER-GUIDE.md b/DEVELOPER-GUIDE.md index c5ec895f0c..ffdd72b3ac 100644 --- a/DEVELOPER-GUIDE.md +++ b/DEVELOPER-GUIDE.md @@ -135,7 +135,7 @@ If you've made changes to `dotnet-interactive` and want to try them out with Vis "[vscode]", "stdio", "--log-path", - "/path/to/a/folder/for/your/logs/", + "/path/to/a/folder/for/your/kernel-logs/", "--verbose", "--working-dir", "{working_dir}" @@ -144,7 +144,9 @@ If you've made changes to `dotnet-interactive` and want to try them out with Vis "dotnet-interactive.notebookParserArgs": [ "{dotnet_path}", "/PATH/TO/REPO/ROOT/artifacts/bin/dotnet-interactive/Debug/net7.0/Microsoft.DotNet.Interactive.App.dll", - "notebook-parser" + "notebook-parser", + "--log-path", + "/path/to/a/folder/for/your/parser-logs/", ] ``` @@ -156,11 +158,11 @@ If you've made changes to `dotnet-interactive` and want to try them out with Vis ### Use a local build of a Polyglot Notebooks extension -If you've made changes to one of the Polyglot Notebooks extensions and want to use them locally, follow these steps: +If you've made changes to the Polyglot Notebooks extension and want to try your changes locally, follow these steps: 1. Run `build.cmd -pack`/`./build.sh --pack` to create the Nuget packages for the extensions -2. Ensure that there aren't any kernels running for the extension in question. It's generally best to close all Notebooks opened in VS Code to accomplish this. +2. Ensure that there aren't any kernels running for the extension in question. It's generally best to close all notebooks opened in VS Code as they might be locking some of these files. 3. Run the `Polyglot Notebook: Create a new blank notebook` command in VS Code. Select `.dib` or `.ipynb` as the extension and any language as default. @@ -174,35 +176,35 @@ If you've made changes to one of the Polyglot Notebooks extensions and want to u - On the `#i` line, fill in the path to the `dotnet-interactive` repo root - On the `#r` line, use the same `EXTENSIONNAME` above, and then look in the `artifacts\packages\Debug\Shipping` folder for the package you're using and get the version number from the name. e.g. a package named `Microsoft.DotNet.Interactive.SqlServer.1.0.0-dev.nupkg` would result in this line `#r "nuget: Microsoft.DotNet.Interactive.SqlServer, 1.0.0-dev"` -```powershell -#!powershell + ```powershell + #!powershell -$FolderName = "\PATH\TO\NUGET\CACHE\packages\microsoft.dotnet.interactive." -if (Test-Path $FolderName) { + $FolderName = "\PATH\TO\NUGET\CACHE\packages\microsoft.dotnet.interactive." + if (Test-Path $FolderName) { - Remove-Item $FolderName -Recurse -Force -} -else -{ - Write-Host "Folder Doesn't Exist" -} + Remove-Item $FolderName -Recurse -Force + } + else + { + Write-Host "Folder Doesn't Exist" + } -#!csharp + #!csharp -#i "nuget: \PATH\TO\REPO\ROOT\artifacts\packages\Debug\Shipping" + #i "nuget: \PATH\TO\REPO\ROOT\artifacts\packages\Debug\Shipping" -#r "nuget: Microsoft.DotNet.Interactive., " -``` + #r "nuget: Microsoft.DotNet.Interactive., " + ``` 7. Run the cell - - If you get an error about access being denied, ensure that all other Notebooks are closed and then restart the kernel again as in step 5 + - If you get an error about access being denied, ensure that all other notebooks are closed and then restart the kernel again as in step 5. 8. Now, use the kernel as you normally would. You should see your local changes being used by the extension. -### Setup full suite of tests to run +### Set up full suite of tests to run -Some tests require additional setup or will be skipped. `JupyterKernel` tests for e.g. are setup to have the same test run against a Jupyter server, directly against the Jupyter kernel over ZMQ and with a simulation of the messages. Jupyter server and Jupyter kernel tests require the following setup or will be skipped while the tests with simulation can be run without additional steps. +Some tests require additional setup or will be skipped. `JupyterKernel` tests for example are set up to have the same test run against a Jupyter server, directly against the Jupyter kernel over ZMQ, and with a simulation of the messages. Jupyter server and Jupyter kernel tests require the following setup or will be skipped. The simulation tests can be run without additional steps. ### Run tests with a local Jupyter Server @@ -224,16 +226,22 @@ jupyter notebook --no-browser --NotebookApp.token= --port=8888 ### Run tests with a Jupyter Kernel over ZMQ 1. Install [Anaconda](https://www.anaconda.com/products/distribution) + 2. [Install R kernel](https://docs.anaconda.com/anaconda/user-guide/tasks/using-r-language/) for R tests by calling the following in Anaconda Prompt (Windows) or the terminal (Mac/Linux) -``` -conda install -c r r-irkernel -``` + + ```console + conda install -c r r-irkernel + ``` + 3. Start Anaconda Bash prompt and create an environment variable `TEST_DOTNET_JUPYTER_ZMQ_CONN` and set to `true` and reactivate your conda environment -```bash -conda env config vars set TEST_DOTNET_JUPYTER_ZMQ_CONN=true -conda activate base -``` -3. Restart `dotnet-interactive.sln` from the Anaconda Bash prompt. + + ```bash + conda env config vars set TEST_DOTNET_JUPYTER_ZMQ_CONN=true + conda activate base + ``` + +3. Restart `dotnet-interactive.sln` from the Anaconda Bash prompt. + 4. The tests will now use the environment variable to connect to your server. ### Run tests directly against the language handler scripts @@ -241,6 +249,8 @@ conda activate base These tests can be run directly against the language handler scripts. This is useful for when making changes on the scripts sent to the jupyter kernel without needing a full integration. 1. Python tests can be run directly in the Anaconda Prompt with IPython by calling `src\Microsoft.DotNet.Interactive.Jupyter.Tests\LanguageHandlerTests\run_python_tests.bat` + 2. R tests can be run directly in the Anaconda Prompt with RScript by calling `src\Microsoft.DotNet.Interactive.Jupyter.Tests\LanguageHandlerTests\run_r_tests.bat` + 3. Both Python and R tests can be run together in the Anaconda Prompt by calling `src\Microsoft.DotNet.Interactive.Jupyter.Tests\LanguageHandlerTests\run_tests.bat` diff --git a/docs/formatting.md b/docs/formatting.md index ba80191307..c04b669bc7 100644 --- a/docs/formatting.md +++ b/docs/formatting.md @@ -15,9 +15,9 @@ Formatters are also used to format the output you see for .NET objects in the Po For any given object, many different string representations are possible. These different representations have associated MIME types, identified by short strings such as `text/html` or `application/json`. MIME types can be used to request specific formatting for an object using the `Display` extension method, which you can call with any object. In this example, we can display a `Rectangle` object assigned to variable `rect` by calling `rect.Display()`: -hmmmm + -Note that the default MIME type in Polyglot Notebooks is `text/html`. This can vary from one type to another, but in the example above, no custom settings have been applied for the `Rectangle` type. (We'll show more about how to do that below.) +Note that the default MIME type in Polyglot Notebooks is `text/html`. This can vary from one .NET type to another, but in the example above, no custom settings have been applied for the `Rectangle` type. (We'll show more about how to do that below.) > _Note: For a cell's return value in C# or F#, only the formatter for the default MIME type can be used._ @@ -100,6 +100,61 @@ Running this code now produces this shorter output:
Submission#3+Node
Next
Submission#3+Node
NextSubmission#3+Node
+ +### Preferred MIME types + +We mentioned above that the default MIME type used for formatting in Polyglot Notebooks is `text/html`. This default is applied when using the `Display()` method without passing a value to the `mimeType` parameter, or when using a `return` statement or trailing expression in C# or F#. This default can be changed globally or for a specific type. + +The following example changes the default for `Rectangle` to `text/plain`. + +```csharp +using System.Drawing; +using Microsoft.DotNet.Interactive.Formatting; + +Formatter.SetPreferredMimeTypesFor(typeof(Rectangle), "text/plain"); + +new Rectangle +{ + Height = 50, + Width = 100 +} +``` + +```console +Rectangle + Location: Point + IsEmpty: True + X: 0 + Y: 0 + Size: Size + IsEmpty: False + Width: 100 + Height: 50 + X: 0 + Y: 0 + Width: 100 + Height: 50 + Left: 0 + Top: 0 + Right: 100 + Bottom: 50 + IsEmpty: False +``` + +You'll also notice that the method can be used to set more than one preferred MIME type. The second parameter is a `params` parameter which allows you to pass multiple values. + +```csharp +Formatter.SetPreferredMimeTypesFor( + typeof(Rectangle), + "text/plain", + "application/json"); +``` + + +If you click on the `...` (`More Actions...`) button to the left of the output and select `Change presentation`, you'll be given a choice of [notebook renderer](https://code.visualstudio.com/api/extension-guides/notebook#notebook-renderer) that you can use to display the different outputs in different ways. + + + ### Replacing the default formatting for a type The default formatters typically show the values of the objects being displayed by printing lists and properties. The output is mostly textual. If you would like to see something different for a given type, whether a different textual output or an image or a plot, you can do this by register custom formatters for specific types. These can be types you defined or types defined in other .NET libraries. One common case where a custom formatter is used is when rendering plots. Some NuGet packages, such as [Plotly.NET](https://www.nuget.org/packages/Plotly.NET/), carry .NET Interactive [extensions](extending-dotnet-interactive.md) that use this feature to provide interactive HTML- and JavaScript-based outputs. @@ -170,45 +225,101 @@ one (254814599) two (656421459) three (-1117028319) #### `TypeFormatterSourceAttribute` -Another approach that can be used to register custom formatters is to decorate a type with `TypeFormatterSourceAttribute`. For use within a notebook, this isn't as convenient as `Formatter.Register`. But if you're writing a .NET Interactive extension or a library or app that uses `Microsoft.DotNet.Interactive.Formatting`, this is the recommended approach. +There is another approach that can be used to register custom formatters. You can decorate a type with `TypeFormatterSourceAttribute`. This isn't the most convenient approach if you want to redefine formatter settings directly within a notebook. But if you're writing a .NET Interactive extension, or a library or app that includes custom formatting for certain types, this is the recommended approach. One reason for this is that the attribute-based approach is simpler. Another reason is that attribute-based formatter customizations are not cleared when `Formatter.ResetToDefault()` is called, while formatters configured using `Formatter.Register` are cleared. You can think of the attribute-based registration approach as a way to set the _default_ formatting for a type. -### Registering preferred MIME types +There are two approaches to attribute-based formatter registration: one for when your project references `Microsoft.DotNet.Interactive.Formatting` and another for when it does not. -We mentioned above that the default MIME type used for formatting in Polyglot Notebooks is `text/html`. This default is applied when using the `Display()` method without passing a value to the `mimeType` parameter or when using a `return` statement or trailing expression in C# or F#. This default can be changed globally or for a specific type. +If you already have a reference to `Microsoft.DotNet.Interactive.Formatting`, for example because you're writing a .NET Interactive extension, then you can decorate your types that need custom formatting with the `TypeFormatterSourceAttribute` defined in `Microsoft.DotNet.Interactive.Formatting`. Here's an example: -The following example changes the default for `Rectangle` to `text/plain`. +```csharp +[TypeFormatterSource(typeof(MyFormatterSource))] +public class MyTypeWithCustomFormatting +{ +} +``` + +A formatter source specified by the `TypeFormatterSourceAttribute` must implement `ITypeFormatterSource` and must have an empty constructor. It does not need to be a public type. Here's an example implementation: ```csharp -using System.Drawing; +internal class MyFormatterSource : ITypeFormatterSource +{ + public IEnumerable CreateTypeFormatters() + { + return new ITypeFormatter[] + { + new PlainTextFormatter(context => + $"Hello from {nameof(MyFormatterSource)} using MIME type text/plain"), + new HtmlFormatter(context => + $"Hello from {nameof(MyFormatterSource)} using MIME type text/html") + }; + } +} +``` -Formatter.SetPreferredMimeTypesFor(typeof(Rectangle), "text/plain"); +As the example above shows, a formatter source can return multiple formatters. These can be formatters for different MIME types and also for different .NET types. -new Rectangle -{ - Height = 50, - Width = 100 +A formatter source can also specify the preferred MIME types for your type: + +```csharp +[TypeFormatterSource( + typeof(MyFormatterSource), + PreferredMimeTypes = new[] { "text/html", "application/json" })] +public class MyTypeWithCustomFormatting +{ } ``` -```console -Rectangle - Location: Point - IsEmpty: True - X: 0 - Y: 0 - Size: Size - IsEmpty: False - Width: 100 - Height: 50 - X: 0 - Y: 0 - Width: 100 - Height: 50 - Left: 0 - Top: 0 - Right: 100 - Bottom: 50 - IsEmpty: False +Specifying several MIME types results in a call to `Formatter.SetPreferredMimeTypesFor`, as described under [Preferred MIME types](#preferred-mime-types). The behavior is equivalent. + +If you don't want to take a dependency on `Microsoft.DotNet.Interactive.Formatting`, there's another to way to provide custom formatting for your types. This can be the simpler approach if you're building a library and you'd like to provide an augmented experience in notebooks but you don't need the other tools available to .NET Interactive extensions, such as magic commands and custom kernels. + +With this approach, you declare your own `TypeFormatterSourceAttribute` and apply it. The shape of the class should match the `TypeFormatterSourceAttribute` defined in `Microsoft.DotNet.Interactive.Formatting`. The `TypeFormatterSourceAttribute` class that you define will be recognized by name. Like the formatter source, it can be declared as internal so that it has no impact on your public API surface. + +Here's the complete definition, which you can copy into your project: + +```csharp +[AttributeUsage(AttributeTargets.Class)] +internal class TypeFormatterSourceAttribute : Attribute +{ + public TypeFormatterSourceAttribute(Type formatterSourceType) + { + FormatterSourceType = formatterSourceType; + } + + public Type FormatterSourceType { get; } + + public string[] PreferredMimeTypes { get; set; } +} +``` + +As with the attribute class itself, the specified formatter source and the formatters it returns are based on naming conventions rather than types. The following is a complete example that you can use as a template: + +```csharp +internal class MyConventionBasedFormatterSource +{ + public IEnumerable CreateTypeFormatters() + { + yield return new MyConventionBasedFormatter { MimeType = "text/html" }; + } +} + +internal class MyConventionBasedFormatter +{ + public string MimeType { get; set; } + + public bool Format(object instance, TextWriter writer) + { + if (instance is MyTypeWithCustomFormatting myObj) + { + writer.Write($"
Custom formattering for {obj}
"); + return true; + } + else + { + return false; + } + } +} ``` ### Resetting formatting configurations @@ -219,7 +330,7 @@ As you experiment with different formatting configurations, you might find you w Formatter.ResetToDefault(); ``` -This will reset all of the configurations that might have been changed by using the APIs described above. Note that this might also reset formatting set up by extensions installed using NuGet packages. +This will reset all of the configurations that might have been changed by using the APIs described above. Note that this might also reset formatting set up by extensions installed using NuGet packages. ## How a formatter is chosen diff --git a/docs/javascript-overview.md b/docs/javascript-overview.md index 97f0a7b00c..8f189e295b 100644 --- a/docs/javascript-overview.md +++ b/docs/javascript-overview.md @@ -153,15 +153,15 @@ Many polyglot notebook workflows use languages such as C#, F#, and SQL to gather var array = new[] { 1, 2, 3 }; ``` -```javascript -#!share --from csharp array +```csharp +#!set --value @javascript:array --name array -return array; +array ``` When you run this code in Polyglot Notebooks, you can see that the `array` variable has been copied to the JavaScript kernel. -image +image You can also share back in the other direction. Let's modify the JavaScript array and share it back to C#. diff --git a/src/polyglot-notebooks-vscode/DEVGUIDE.md b/src/polyglot-notebooks-vscode/DEVGUIDE.md index ccb19c58f6..5ea82ded32 100644 --- a/src/polyglot-notebooks-vscode/DEVGUIDE.md +++ b/src/polyglot-notebooks-vscode/DEVGUIDE.md @@ -3,22 +3,43 @@ The organization of this extension is a bit complicated. === -This repo depends on symbolic links between directories. By default Windows doesn't support this feature. To work around this scenario, please run the PowerShell script `/src/ensure-symlinks.ps1` as an administrator. This usually only needs to be run once. +This repo depends on symbolic links between directories. By default Windows doesn't support this feature. To work around this scenario, please run the PowerShell script `/src/ensure-symlinks.ps1` as an administrator. This usually only needs to be run once. If you run `git clean` you will need to repeat this step. -## tl;dr - +## Setup steps 0. Requirements: - - nodejs v12.16.1. - - npm v6.14.11. - - git **with symlink support** - _or_ - - Latest stable [PowerShell 7](https://github.com/PowerShell/PowerShell/releases/) on the path - _and_ - [Developer Mode](https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development) enabled for Windows. + * nodejs v12.16.1. + * npm v6.14.11. + * Shell requirements: + * git **with symlink support** + + **_or_** + + * Latest stable [PowerShell 7](https://github.com/PowerShell/PowerShell/releases/) on the path + + **_and_** + + * [Developer Mode](https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development) enabled for Windows. + 1. Open a terminal in either the `src/polyglot-notebooks-vscode/` or `src/polyglot-notebooks-insiders/` directory. + 2. `npm install` -3. Open the appropriate VS Code, e.g., for stable: `code .`, for insiders: `code-insiders .` -4. F5. + +3. Open the appropriate VS Code. + + For VS Code stable run: + + ```console + code . + ``` + + For VS Code Insiders run: + + ```console + code-insiders . + ``` + +4. Press F5. ## The complicated bits diff --git a/src/polyglot-notebooks-vscode/README.md b/src/polyglot-notebooks-vscode/README.md index e0b1467d5d..59f0c7f5f6 100644 --- a/src/polyglot-notebooks-vscode/README.md +++ b/src/polyglot-notebooks-vscode/README.md @@ -2,6 +2,8 @@ The [Polyglot Notebooks extension](https://marketplace.visualstudio.com/items?itemName=ms-dotnettools.dotnet-interactive-vscode), powered by [.NET Interactive](https://github.com/dotnet/interactive), brings support for multi-language notebooks to Visual Studio Code. Classic notebook software typically supports notebooks that use only one language at a time. With Polyglot Notebooks, features such as completions, documentation, syntax highlighting, and diagnostics are available for many languages in one notebook. In addition, different cells in the same notebook can run in separate processes or on different machines, allowing a notebook to span local and cloud environments in one combined workflow. +Read more in our [FAQ](https://github.com/dotnet/interactive/blob/main/docs/FAQ.md). + ## Fully Interoperable with Jupyter Polyglot Notebooks are fully interoperable with Jupyter and support the `.ipynb` file extension. You don't need to choose between the capabilities of Polyglot Notebooks and the rich Jupyter ecosystem. If your notebook is saved in the `.ipynb` format, you can open it in Jupyter and the cell languages will still be recognized. When working in Jupyter using the .NET Interactive kernel, you can switch cell languages using magic commands. @@ -10,18 +12,18 @@ Polyglot Notebooks are fully interoperable with Jupyter and support the `.ipynb` The following languages are supported by Polyglot Notebooks: -| Language | Variable sharing supported | +| Language | Variable sharing supported | |---------------------------------------------|-------------------------------------------------------| -| C# | ✅ | -| F# | ✅ | -| PowerShell | ✅ | -| JavaScript | ✅ | -| SQL | ✅ | -| KQL ([Kusto Query Language](https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/)) | ✅ | -| HTML | ⛔ | -| [Mermaid](https://mermaid.js.org/intro/) | ⛔ | -| [Python](https://github.com/dotnet/interactive/blob/main/docs/jupyter-in-polyglot-notebooks.md) | ✅ | -| [R](https://github.com/dotnet/interactive/blob/main/docs/jupyter-in-polyglot-notebooks.md) | ✅ | +| C# | ✅ | +| F# | ✅ | +| PowerShell | ✅ | +| JavaScript | ✅ | +| SQL | ✅ | +| KQL ([Kusto Query Language](https://learn.microsoft.com/en-us/azure/data-explorer/kusto/query/)) | ✅ | +| HTML | ⛔ | +| [Mermaid](https://mermaid.js.org/intro/) | ⛔ | +| [Python](https://github.com/dotnet/interactive/blob/main/docs/jupyter-in-polyglot-notebooks.md) (Preview) | ✅ | +| [R](https://github.com/dotnet/interactive/blob/main/docs/jupyter-in-polyglot-notebooks.md) (Preview) | ✅ | ## Features