Add Engineering Plan for Linux Distros #789
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add Engineering Plan for including .NET Core in Linux Distro Repositories skip ci please
shawnro
reviewed
Sep 25, 2018
| Each distro has specific packaging guidelines along with a well-defined review process. These guidelines are similar across distros, with one of the main guidelines being that the source provided contain no pre-built binaries. This is the biggest obstacle to getting .NET Core into distro archives. .NET Core is included in the Red Hat distro today even though the source-build process requires a large number of pre-builts in order to successfully build. Red Hat has given exceptions for us to be able to be included in their distro. | ||
|
|
||
| ### Toolset for Building .NET Core ### | ||
| There is one exception to the *"no prebuilt binaries rule"* for the case of bootstrapping a toolset. This is an exception that .NET Core will need to take advantage of. This boostrap process is illustrated below. An exsting toolset is used to build a toolset from source. The existing toolset is then thrown away and the newly built toolset is used for the actual build of the assemblies that go into the archive. The source is the same for each stage of the build. |
omajid
reviewed
Sep 25, 2018
| 3. Prebuilts that have a version built with source-build - The source-build process should be modified to get repos to use the source-built version of the package. Many of these are older | ||
| 4. Prebuilts containing reference assemblies - These prebuilts can be delivered as IL-only code and re-compiled as reference assemblies. | ||
| 5. First party prebuilts - If there are dependencies on 1st party packages that are not built as part of source-build, we should investigate including them in source-build if possible or eliminating the dependency. | ||
| 6. Third party prebuilts - Steps should be taken to remove any dependencies on 3rd party packages. |
There was a problem hiding this comment.
Another possible way to deal with 5 and 6 is to package them up as separate (distro) packages. When building for a distro, it should be perfectly fine to use a dependency that's available in the distribution. This is the case for native code already (clang, lldb, lttng, openssl). Theoretically, we should be able to do the same with .NET packages as well. We could then have a dependency on the already built packages.
dagood
reviewed
Sep 25, 2018
| ### Toolset for Building .NET Core ### | ||
| There is one exception to the *"no prebuilt binaries rule"* for the case of bootstrapping a toolset. This is an exception that .NET Core will need to take advantage of. This boostrap process is illustrated below. An exsting toolset is used to build a toolset from source. The existing toolset is then thrown away and the newly built toolset is used for the actual build of the assemblies that go into the archive. The source is the same for each stage of the build. | ||
|
|
||
| Toolset bootstrapping is different than runtime bootstapping provided in https://github.com/dotnet/source-build/blob/master/scripts/bootstrap/buildbootstrapcli.sh. This script is only needed when starting with a new distro on which .NET Core has never built. For the distros being discussed in this document, there are already existing .NET Core runtime and SDK builds available. For that reason, runtime bootstrapping is outside the scope of this document. |
There was a problem hiding this comment.
Pin URL to commit to avoid 404 in case the sh script moves? Maybe shorten link like scripts/bootstrap/buildbootstrapcli.sh?
| ### Toolset for Building .NET Core ### | ||
| There is one exception to the *"no prebuilt binaries rule"* for the case of bootstrapping a toolset. This is an exception that .NET Core will need to take advantage of. This boostrap process is illustrated below. An exsting toolset is used to build a toolset from source. The existing toolset is then thrown away and the newly built toolset is used for the actual build of the assemblies that go into the archive. The source is the same for each stage of the build. | ||
|
|
||
| Toolset bootstrapping is different than runtime bootstapping provided in https://github.com/dotnet/source-build/blob/master/scripts/bootstrap/buildbootstrapcli.sh. This script is only needed when starting with a new distro on which .NET Core has never built. For the distros being discussed in this document, there are already existing .NET Core runtime and SDK builds available. For that reason, runtime bootstrapping is outside the scope of this document. |
There was a problem hiding this comment.
This paragraph doesn't feel like it's in the right place. The next paragraph picks up with "This is discussed in the Fedora packaging guidelines as an exception...", and at first I think "This" refers to runtime bootstrapping but after reading the full sentence it's referring to toolset boostrapping.
Maybe move this after the image, even put it in a block quote to distinguish it as a clarification?
|
|
||
| ### Runtime vs. SDK ### | ||
|
|
||
| In the requirements doc above, it is suggested that we *Reduce the problem space and get in the door. Deliver the runtime first.* The full toolset is required to build both the runtime and the SDK. Delivering runtime first doesn't necessarily reduce the problem space because of this requirement to create a bootstrap toolset, first. |
There was a problem hiding this comment.
Consider making the requirements doc a "reference-style link" so you can link it in both places without duplicating the github issue URL.
|
|
||
| ## Work Breakdown ## | ||
| With this partnership, there are four main area of focus to get .NET Core ready for inclusion into Fedora's package archive: | ||
| 1. Prove out and define process for toolset bootstrapping.(https://github.com/dotnet/source-build/issues/724) |
There was a problem hiding this comment.
Missing space in bootstrapping.(https
| Source-build changes will be ported to `release/2.2` and `master` on a regular basis to keep those branches up to date. The desire is to have `release/2.2` and `master` updated and ready to be included in other distros when they are ready to be released. | ||
|
|
||
| ### Toolset Bootstrapping ## | ||
| This work requires setting up scripts to build the .NET Core toolset from source and then using the result to build source-build. This is to define the process and to ensure that it works consistently by adding CI. It also will require working out issues that we've encountered building source-build with the same version that is being built. See https://github.com/dotnet/source-build/issues/606. This will also track work with source-building buildTools and the eventual move to Arcade, if it is ported to the `release/2.1` branch for any repos. |
There was a problem hiding this comment.
source-building buildTools => source-building [dotnet/BuildTools](https://github.com/dotnet/buildtools)? (I included dotnet/ because there's an aspnet/buildtools too.)
Link for Arcade, too?
| This work requires setting up scripts to build the .NET Core toolset from source and then using the result to build source-build. This is to define the process and to ensure that it works consistently by adding CI. It also will require working out issues that we've encountered building source-build with the same version that is being built. See https://github.com/dotnet/source-build/issues/606. This will also track work with source-building buildTools and the eventual move to Arcade, if it is ported to the `release/2.1` branch for any repos. | ||
|
|
||
| ### Runtime / SDK Split ### | ||
| The main part of this work is to split and isolate the runtime and sdk builds into separate directories with separate build scripts in source-build. The result of the runtime build should be input to the sdk build. Each should be able to be built independently. |
There was a problem hiding this comment.
I'm not sure separate directories is the best way to handle this, but I think this is just an implementation detail. Maybe leave that specific bit out from this doc and link to an issue here?
| ### Prebuilt Binary Elimination ### | ||
|
|
||
| There are 6 categories of prebuilts to eliminate: | ||
| 1. Prebuilts that end up in the built product. - These must be eliminated and a process must be put into place to detect and track these to make sure they don't happen again. |
There was a problem hiding this comment.
I think this list would be easier to read if the subpoints were put into nested bullets, or the category name bolded:
- Prebuilts that end up in the built product.
- These must be eliminated and a process must be put into place to detect and track these to make sure they don't happen again.
- Prebuilts loaded in unneccessary functionality
- These are prebuilts that are loaded by a repo's source even though the functionality for the source is not included in source-build. The main group of these are test assemblies, since source-build does not build tests.
- Prebuilts that have a version built with source-build
- The source-build process should be modified to get repos to use the source-built version of the package. Many of these are older
| There are 6 categories of prebuilts to eliminate: | ||
| 1. Prebuilts that end up in the built product. - These must be eliminated and a process must be put into place to detect and track these to make sure they don't happen again. | ||
| 2. Prebuilts loaded in unneccessary functionality - These are prebuilts that are loaded by a repo's source even though the functionality for the source is not included in source-build. The main group of these are test assemblies, since source-build does not build tests. | ||
| 3. Prebuilts that have a version built with source-build - The source-build process should be modified to get repos to use the source-built version of the package. Many of these are older |
There was a problem hiding this comment.
Missing period, or incomplete sentence? "Many of these are older"
| 1. Prebuilts that end up in the built product. - These must be eliminated and a process must be put into place to detect and track these to make sure they don't happen again. | ||
| 2. Prebuilts loaded in unneccessary functionality - These are prebuilts that are loaded by a repo's source even though the functionality for the source is not included in source-build. The main group of these are test assemblies, since source-build does not build tests. | ||
| 3. Prebuilts that have a version built with source-build - The source-build process should be modified to get repos to use the source-built version of the package. Many of these are older | ||
| 4. Prebuilts containing reference assemblies - These prebuilts can be delivered as IL-only code and re-compiled as reference assemblies. |
There was a problem hiding this comment.
Prebuilts containing only reference assemblies?
| * Be visible and accessible to source-build team members, partners and management. | ||
| * Identify the category of prebuilt, if possible. | ||
|
|
||
| ### Coordidinate with DotNet SIG ### |
There was a problem hiding this comment.
Typo: "Coordidinate" => "Coordinate"
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add Engineering Plan for including .NET Core in Linux Distro Repositories
skip ci please