Quantcast
Channel: .NET Blog
Viewing all 4000 articles
Browse latest View live

.NET Framework February 2019 Security and Quality Rollup

$
0
0

Updated: February 19, 2019

  • The advisory is now resolved with no action needed from Microsoft Customers. The issue was not applicable to any valid or supported configuration. There is no consequence for .NET 4.8 Preview customers.  We strive to share timely information to protect our customer’s productivity, in this case, our finding was thankfully of no consequence for customers on supported configurations.

Updated: February 15, 2019

  • A new advisory has been released today for issues customers have reported with .NET 4.8 Preview and this security update for Windows 10 update 1809 installed.

Updated: February 14, 2019

  • Added the previously released previous release update information in quality and reliability for easier reference.

 

Yesterday, we released the February 2019 Security and Quality Rollup.

Security

CVE-2019-0613 – Remote Code Execution Vulnerability

This security update resolves a vulnerability in .NET Framework software if the software does not check the source markup of a file. An attacker who successfully exploits the vulnerability could run arbitrary code in the context of the current user. If the current user is logged on by using administrative user rights, an attacker could take control of the affected system. An attacker could then install programs; view, change, or delete data; or create new accounts that have full user rights. Users whose accounts are configured to have fewer user rights on the system could be less affected than users who have administrative user rights.

CVE-2019-0657 – Domain Spoofing Vulnerability

This security update resolves a vulnerability in certain .NET Framework APIs that parse URLs. An attacker who successfully exploits this vulnerability could use it to bypass security logic that’s intended to make sure that a user-provided URL belonged to a specific host name or a subdomain of that host name. This could be used to cause privileged communication to be made to an untrusted service as if it were a trusted service.

CVE-2019-0657

Quality and Reliability

This release contains the following previously released quality and reliability improvements, January 2019 Preview of Quality Rollup.

Getting the Update

The Security and Quality Rollup is available via Windows Update, Windows Server Update Services, Microsoft Update Catalog, and Docker.

Microsoft Update Catalog

You can get the update via the Microsoft Update Catalog. For Windows 10 1507, Windows 10 update 1607, Windows 10 update 1703, Windows 10 update 1709 and Windows update 1803, the .NET Framework updates are part of the Windows 10 Monthly Rollup.  For Windows 10 update 1809 the .NET Framework updates are part of the Cumulative Update for .NET Framework which is not part of the Windows 10 Monthly Rollup.

The following table is for Windows 10 and Windows Server 2016+ versions.

Product Version Security and Quality Rollup KB
Windows 10 1809 (October 2018 Update)
Windows Server 2019
Catalog
4483452
.NET Framework 3.5, 4.7.2 4483452
Windows 10 1803 (April 2018 Update) Catalog
4487017
.NET Framework 3.5, 4.7.2 4487017
Windows 10 1709 (Fall Creators Update) Catalog
4486996
.NET Framework 3.5, 4.7.1, 4.7.2 4486996
Windows 10 1703 (Creators Update) Catalog
4487020
.NET Framework 3.5, 4.7, 4.7.1, 4.7.2 4487020
Windows 10 1607 (Anniversary Update)
Windows Server 2016
Catalog
4487026
.NET Framework 3.5, 4.6.2, 4.7, 4.7.1, 4.7.2 4487026
Windows 10 1507 Catalog
4487018
.NET Framework 3.5, 4.6, 4.6.1, 4.6.2 4487018

The following table is for earlier Windows and Windows Server versions.

Product Version Security and Quality Rollup KB Security Only Update KB
Windows 8.1
Windows RT 8.1
Windows Server 2012 R2
Catalog
4487080
Catalog
4487124
.NET Framework 3.5 Catalog
4483459
Catalog
4483484
.NET Framework 4.5.2 Catalog
4483453
Catalog
4483472
.NET Framework 4.6, 4.6.1, 4.6.2, 4.7, 4.7.1, 4.7.2 Catalog
4483450
Catalog
4483469
Windows Server 2012 Catalog
4487079
Catalog
4487122
.NET Framework 3.5 Catalog
4483456
Catalog
4483481
.NET Framework 4.5.2 Catalog
4483454
Catalog
4483473
.NET Framework 4.6, 4.6.1, 4.6.2, 4.7, 4.7.1, 4.7.2 Catalog
4483449
Catalog
4483468
Windows 7 SP1
Windows Server 2008 R2 SP1
Catalog
4487078
Catalog
4487121
.NET Framework 3.5.1 Catalog
4483458
Catalog
4483483
.NET Framework 4.5.2 Catalog
4483455
Catalog
4483474
.NET Framework 4.6, 4.6.1, 4.6.2, 4.7, 4.7.1, 4.7.2 Catalog
4483451
Catalog
4483470
Windows Server 2008 Catalog
4487081
Catalog
4487124
.NET Framework 2.0, 3.0 Catalog
4483457
Catalog
4483482
.NET Framework 4.5.2 Catalog
4483455
Catalog
4483474
.NET Framework 4.6 Catalog
4483451
Catalog
4483470

Docker Images

We are updating the following .NET Framework Docker images for today’s release:

Note: Look at the “Tags” view in each repository to see the updated Docker image tags.

Note: Significant changes have been made with Docker images recently. Please look at .NET Docker Announcements for more information.

Previous Monthly Rollups

The last few .NET Framework Monthly updates are listed below for your convenience:


Help us make the .NET Architecture guides better for you!

$
0
0

Over the last couple of years, we worked with experts to create some incredible architecture guides & reference samples for .NET developers. We focused on Microservices Architecture, Modernizing existing .NET apps, DevOps best practices, ASP.NET web apps, Azure cloud apps, Mobile apps with Xamarin and more.

We’re running a survey to find out how you’re using them and what we could improve. It should take 10 mins or less, and your answers will help us make the guides and reference samples better.


Take the survey now! 

Thank You!
.NET Team

Microsoft’s Developer Blogs are Getting an Update

$
0
0

In the coming days, we’ll be moving our developer blogs to a new platform with a modern, clean design and powerful features that will make it easy for you to discover and share great content. This week, you’ll see the Visual Studio, IoTDev, and Premier Developer blogs move to a new URL  – while additional developer blogs will transition over the coming weeks. You’ll continue to receive all the information and news from our teams, including access to all past posts. Your RSS feeds will continue to seamlessly deliver posts and any of your bookmarked links will re-direct to the new URL location.

We’d love your feedback

Our most inspirational ideas have come from this community and we want to make sure our developer blogs are exactly what you need. Share your thoughts about the current blog design by taking our Microsoft Developer Blogs survey and tell us what you think! We would also love your suggestions for future topics that our authors could write about. Please use the survey to share what kinds of information, tutorials, or features you would like to see covered in future posts.

Frequently Asked Questions

For the most-asked questions, see the FAQ below. If you have any additional questions, ideas, or concerns that we have not addressed, please let us know by submitting feedback through the Developer Blogs FAQ page.

Will Saved/Bookmarked URLs Redirect?

Yes. All existing URLs will auto-redirect to the new site. If you discover any broken links, please report them through the FAQ feedback page.

Will My Feed Reader Still Send New Posts?

Yes, your RSS feed will continue to work through your current set-up. If you encounter any issues with your RSS feed, please report it with details about your feed reader.

Which Blogs Are Moving?

This migration involves the majority of our Microsoft developer blogs so expect to see changes to our Visual Studio, IoTDev, and Premier Developer blogs this week. The rest will be migrated in waves over the next few weeks.

We appreciate all of the feedback so far and look forward to showing you what we’ve been working on! For additional information about this blog migration, please refer to our Developer Blogs FAQ.

RESOLVED: Advisory on February 2019 Security update for Windows 10 update 1809

$
0
0

Final Update 2/19/19 @1:30 PM (PST): This advisory is now resolved with no action needed from Microsoft Customers. The issue was not applicable to any valid or supported configuration. There is no consequence for .NET 4.8 Preview customers.

We strive to share timely information to protect our customer’s productivity, in this case, our finding was thankfully of no consequence for customers on supported configurations.

Update 2/15/19 @3:35 PM (PST): As we continue our investigation, we are finding the issue to be restricted to a limited and isolated set of test-only systems that are using non-official versions of the .NET 4.8 Preview. As of 2/15/19 around 12:00 pm (PST) we further tightened our delivery mechanisms to ensure that the February .NET security updates are only deployed to their expected target systems.

The February 2019 Security and Quality Rollup for Windows 10 update 1809 was released earlier this week. We have received multiple customer reports of issues when a customer has installed .NET Framework 4.8 Preview then installed the February security update, 4483452. These reports are specific to only to the scenario when a customer has installed both .NET Framework 4.8 Preview and the February security update.

We are actively working on investigating and addressing this issue. If you installed the February 2019 security update and have not seen any negative behavior, we recommend that you leave your system as-is but closely monitor it and ensure that you apply upcoming .NET Framework updates.

We will continue to update this post as we have new information.

Guidance

We are working on guidance and will update this post and as we have new information.

Workaround

There are no known workarounds at this time.

Symptoms

After installing .NET Framework 4.8 Preview then the February 2019 security update, 4483452, Visual Studios will crash with this error message:

“Exception has been thrown by the target of an invocation.”

.NET Core 1.0 and 1.1 will reach End of Life on June 27, 2019

$
0
0

.NET Core 1.0 was released on June 27, 2016 and .NET Core 1.1 was released on November 16, 2016. As an LTS release, .NET Core 1.0 is supported for three years. .NET Core 1.1 fits into the same support timeframe as .NET Core 1.0. .NET Core 1.0 and 1.1 will reach end of life and go out of support on June 27, 2019, three years after the initial .NET Core 1.0 release.

After June 27, 2019, .NET Core patch updates will no longer include updated packages or container images for .NET Core 1.0 and 1.1. You should plan your upgrade from .NET Core 1.x to .NET Core 2.1 or 2.2 now.

Upgrade to .NET Core 2.1

The supported upgrade path for .NET Core 1.x applications is via .NET Core 2.1 or 2.2. Instructions for upgrading can be found in the following documents:

Note: The migration documents are written for .NET Core 2.0 to 2.1 migration, but equally apply to .NET Core 1.x to 2.1 migration.

.NET Core 2.1 is a long-term support (LTS) release. We recommend that you make .NET Core 2.1 your new standard for .NET Core development, particularly for apps that are not updated often.

.NET Core 2.0 has already reached end-of-life, as of October 1, 2018. It is important to migrate applications to at least .NET Core 2.1.

Microsoft Support Policy

Microsoft has a published support policy for .NET Core. It includes policies for two release types: LTS and Current.

.NET Core 1.0, 1.1 and 2.1 are LTS releases. .NET Core 2.2 is a Current release.

  • LTS releases are designed for long-term support. They included features and components that have been stabilized, requiring few updates over a longer support release lifetime. These releases are a good choice for hosting applications that you do not intend to update.
  • Current releases include new features that may undergo future change based on feedback. These releases are a good choice for applications in active development, giving you access to the latest features and improvements. You need to upgrade to later .NET Core releases more often to stay in support.

Both types of releases receive critical fixes throughout their lifecycle, for security, reliability, or to add support for new operating system versions. You must stay up to date with the latest patches to qualify for support.

The .NET Core Supported OS Lifecycle Policy defines which Windows, macOS and Linux versions are supported for each .NET Core release.

.NET Framework February 2019 Preview of Quality Rollup

$
0
0

Today, we released the February 2019 Preview of Quality Rollup.

Quality and Reliability

This release contains the following quality and reliability improvements.

CLR

  • Addresses an issue in System.Threading.Timer where a single global queue that was protected by a single process-wide lock causing a issues with scalability where Timers are used frequently on multi-CPU machine.  The fix can be opted into with an AppContext switch below.  See instructions for enabling the switch.  [750048]
    • Switch name: Switch.System.Threading.UseNetCoreTimer
    • Switch value to enable: true
      Don’t rely on applying the setting programmatically – the switch value is read only once per AppDomain at the time when the System.Threading.Timer type is loaded.

SQL

  • Addresses an issue that caused compatibility breaks seen in some System.Data.SqlClient usage scenarios. [721209]

WPF

  • Improved the memory allocation and cleanup scheduling behavior of the weak-event pattern.   The fix can be opted into with an AppContext switch below.  See instructions for enabling the switch.  [676441]
    • Switch name: Switch.MS.Internal.EnableWeakEventMemoryImprovements
    • Switch name: Switch.MS.Internal.EnableCleanupSchedulingImprovements
    • Switch value to enable: true

 

Getting the Update

The Preview of Quality Rollup is available via Windows Update, Windows Server Update Services, and Microsoft Update Catalog.

Microsoft Update Catalog

You can get the update via the Microsoft Update Catalog. For Windows 10 update 1607, Windows 10 update 1703, Windows 10 update 1709 and Windows update 1803, the .NET Framework updates are part of the Windows 10 Monthly Rollup.

The following table is for Windows 10 and Windows Server 2016+ versions.

Product Version Cumulative Update KB
Windows 10 1803 (April 2018 Update) Catalog
4487029
.NET Framework 3.5, 4.7.2 4487029
Windows 10 1709 (Fall Creators Update) Catalog
4487021
.NET Framework 3.5, 4.7.1, 4.7.2 4487021
Windows 10 1703 (Creators Update) Catalog
4487011
.NET Framework 3.5, 4.7, 4.7.1, 4.7.2 4487011
Windows 10 1607 (Anniversary Update)
Windows Server 2016
Catalog
4487006
.NET Framework 3.5, 4.6.2, 4.7, 4.7.1, 4.7.2 4487006

The following table is for earlier Windows and Windows Server versions.

Product Version Preview of Quality Rollup KB
Windows 8.1
Windows RT 8.1
Windows Server 2012 R2
Catalog
4487258
.NET Framework 3.5 Catalog
4483459
.NET Framework 4.5.2 Catalog
4483453
.NET Framework 4.6, 4.6.1, 4.6.2, 4.7, 4.7.1, 4.7.2 Catalog
4486545
Windows Server 2012 Catalog
4487257
.NET Framework 3.5 Catalog
4483456
.NET Framework 4.5.2 Catalog
4483454
.NET Framework 4.6, 4.6.1, 4.6.2, 4.7, 4.7.1, 4.7.2 Catalog
4486544
Windows 7 SP1
Windows Server 2008 R2 SP1
Catalog
4487256
.NET Framework 3.5.1 Catalog
4483458
.NET Framework 4.5.2 Catalog
4483455
.NET Framework 4.6, 4.6.1, 4.6.2, 4.7, 4.7.1, 4.7.2 Catalog
4486546
Windows Server 2008 Catalog
4487259
.NET Framework 2.0, 3.0 Catalog
4483457
.NET Framework 4.5.2 Catalog
4483455
.NET Framework 4.6 Catalog
4486546

Previous Monthly Rollups

The last few .NET Framework Monthly updates are listed below for your convenience:

GC Perf Infrastructure – Part 0

$
0
0

In this blog entry and some future ones I will be showing off functionalities that our new GC perf infrastructure provides. Andy and I have been working on it (he did all the work; I merely played the consultant role). We will be open sourcing it soon and I wanted to give you some examples of using it and you can add these to your repertoire of perf analysis techniques when it’s available.

The general GC perf analysis flow on a customer scenario usually goes like this –

1) get a perf trace (ETW trace on Windows and event trace on Linux);

2A) if the customer has no complains and just wanted to see if they can improve things, we first get a general idea of things and see if/how they can be improved or

2B) if the customer does have specific complains (eg, long GC pauses, or too much memory used) we look for things that could cause them.

Of course, as any experienced perf person would know, perf analysis can vary greatly from one case to the next. You look at some data to get some clues to identify the suspicious areas and focus on those areas and get more clues…and usually make incremental progress before you get to the root cause.

To give some context, for data we get from trace, we have a library named TraceEvent that parses it into TraceGC objects. Since GC is per process, each process (that observed at least one GC) will get its list of these TraceGC objects. And the TraceGC type includes information on this GC such as

  • Basic things which are read directly from some GC event’s fields like Number (index of this GC), Generation (which generation this GC collected), PerHeapHistories (which includes a lot of info such as what condemned reasons this heap incurred, the generation data for each generation for this heap)
  • Processed info like PauseDurationMSec (for ephemeral GCs this is the difference between the timestamp of the SuspendEEStart event and the RestartEEStop event);
  • Info that gets light up when you have the additional events, eg, GCCpuMSec if you have CPU samples collected in the trace;

So given a basic GC trace there’s already a ton of info you can get. We do some processing in TraceEvent and a perf analysis means looking at info these TraceGC objects give you in really any number of ways.

If we have a big trace, the time that it took to parse the trace to get these TraceGC objects can be very long. And if I change my analysis code I’d have to start my analysis process again which means I’d have to parse the trace again. This seemed very inefficient. I started searching for options that could persist the TraceGC objects and I can just modify my code to consume them without having to reprocess the trace. And I found Jupyter Notebook which allows you to edit python code in individual cells but the results from them persist in memory and can be used by any other cells. And I found pythonnet which allows you to interop with a c# lib from python. This means I could persist the TraceGC objects in memory and edit the code that I want to look at any of these objects in any way I desire and don’t need to reprocess the trace at all. This along with the nice charting capability from python gave me exactly what I needed. So what I would do is to have one cell that just did the trace processing with all the resulting TraceGC objects, and other cells for various ways to look at info in these objects and edit them whenever I needed.

This was several years ago and I’ve been using it since. When Andy joined the team and started working on a new GC perf infra I asked him to adapt this as part of our infra. And today I looked at a trace with him and below is what we did.

In this case I just got the trace from a customer to see if we can find anything can be improved by something the customer can do or we did but in a release the customer hasn’t upgraded to, or we are doing/planning to do. To start with, we looked at 3 metrics – individual GC pause times (PauseDurationMSec), GC speed PromotedMBPerSec(ie, # of MB GC promoted / (PauseDurationMSec / 1000)) and  heap size in MB after each GC (HeapSizeAfterMB), just to get a general idea. We used the histogram charts for these:

 

*NGC means NonConcurrent GC, I didn’t want to call it BGC (Blocking GC) because we already have BGC mean Background GC.

If you look at the PauseDurationMSec charts for gen0 GCs (NGC0), gen1 GCs (NGC1) and BGCs (there were no full blocking GCs in this trace), most of them were in the range of a few to 20ms. But there are definitely some longer ones, eg some that are between 75 and 100ms in the NGC0 chart. And right off the bat we see some outliers for BGC – most of them are < 40ms but then there are some > 100ms! And it’s hard to see on the charts but there are actually some thin blue lines in the > 100ms range in both the NGC0 and NGC1 charts.

Since we are using Jupyter, we just changed the code in the cell for this and only showed GCs with the PauseDurationMSec > 50ms and redrew the charts – now it’s very clear there are some ephemeral GCs > 100ms pauses and we can see the 1 long BGC pause is 114.2ms.

 

And we can see the GC speed (the PromotedMBPerSec charts) for many of these are low. In the NGC0 chart most of the GCs are on the left hand side with the lowest speed.

If the long GCs’s PromotedMBPerSec was not so low it would mean they simply had a lot more memory to promote which would indicate a GC tuning problem – one very likely reason would be we are not setting the allocation budgets correctly.

But since that’s not the case we wanted to see why these GC’s speed was so low – we spent a long time paused but GC was not able to do work at its normal speed.

Let’s concentrate on the gen0 GCs (NGC0) first as a starting point. We know the way PauseDurationMSec is calculated so it consists of suspending EE (SuspendDurationMSec)+ actual GC work + resuming EE. Resuming the EE generally takes very little time so I’ll not look at it first. I wanted to see if suspension was too long. So we looked at NGC0’s pause and suspension with our table printing function. Since it’s so easy we’ll throw in the total promoted mb and the GC speed and let’s sort by PauseDurationMSec (highest first):

 

“pause msec” is PauseDurationMSec.

“suspend msec” is SuspendDurationMSec.

“promoted mb” is PromotedMB for this GC.

(I’m only showing the top few for brevity)

Right away we see some long suspension times – the GCs that took 156ms and 101ms, spent 95.4ms and 49.4ms in suspension respectively. So that definitely shows a suspension issue which means the suspension in EE had trouble suspending the managed threads. But the other GCs, like the longest one that took 187ms spent very little time in suspension.

We do have another field in the TraceGC class called DurationMSec which is the difference between the timestamp for the GCStart event and the GCStop event. From the first glance this should just be PauseDurationMSec – suspending EE – resuming EE. Almost – there’s a bit of work we have to do between SuspendEEStop and GCStart, and between GCStop and RestartEEStart. So if things work as expected, (PauseDurationMSec – DurationMSec) should be almost the same as (suspending EE + resuming EE). We changed the code again to add a DurationMSec column (“duration msec”) and we sort by that column

 

The longest GC (187ms) has only 8.73ms DurationMSec! And Suspend only took 0.0544ms. So there’s a huge difference between PauseDurationMSec and DurationMSec, not accounted by the suspension cost.

We modified the code again to add a few more columns, mainly the “pause to start” column which is the difference between the timestamp of SuspendEEStart and GCStart so it includes the suspension time. We also calculate a “pause %” column which is (“suspend msec” / “pause to start” * 100) and a “suspend %” column which is (“suspend msec” / “pause msec” * 100). Also we changed the “promoted mb/sec” column to use DurationMSec instead of PauseDurationMSec. Now some rows in the table change very drastically. The table is sorted by the “pause %” column.

 

The GC that took 187ms spent 178ms from SuspendEEStart to GCStart!!! And of course the GC speed (promoted mb/sec) is now a lot higher.

This is enough evidence to tell us that the GC threads are getting severely interfered (suspension is not the only part that was affected), could be from other processes or other threads in this process. We’d need to collect more events to diagnose further.

 

The post GC Perf Infrastructure – Part 0 appeared first on .NET Blog.

.NET Core September 2019 Updates – 2.1.13 and 2.2.7

$
0
0

Today, we are releasing the .NET Core September 2019 Update. These updates contain security and reliability fixes. See the individual release notes for details on updated packages.

NOTE: If you are a Visual Studio user, there are MSBuild version requirements so use only the .NET Core SDK supported for each Visual Studio version. Information needed to make this choice will be seen on the download page. If you use other development environments, we recommend using the latest SDK release.

Security

CVE-2019-1302: ASP.NET Core Elevation Of Privilege Vulnerability

Microsoft is releasing this security advisory to provide information about a vulnerability in ASP.NET Core. This advisory also provides guidance on what developers can do to update their applications to remove this vulnerability.

Microsoft is aware of an elevation of privilege vulnerability exists when an ASP.NET Core web application, created using vulnerable project templates, fails to properly sanitize web requests. An attacker who successfully exploited this vulnerability could perform content injection attacks and run a script in the security context of the logged-on user.

To exploit the vulnerability, an attacker could send a specially crafted email, containing a malicious link, to a user. Alternatively, an attacker could use a chat client to social engineer a user into clicking the malicious link. However, in all cases to exploit this vulnerability, a user must click a maliciously crafted link from an attacker.

The update addresses the vulnerability by correcting how the .NET Core web application handles content encoding and updates the application templates to depend on the corrected code libraries.

CVE-2019-1301: Denial of Service Vulnerability in .NET Core

Microsoft is releasing this security advisory to provide information about a vulnerability in .NET Core. This advisory also provides guidance on what developers can do to update their applications to remove this vulnerability.

Microsoft is aware of a denial of service vulnerability when .NET Core improperly handles web requests. An attacker who successfully exploited this vulnerability could cause a denial of service against a .NET Core web application. The vulnerability can be exploited remotely, without authentication.

The update addresses the vulnerability by correcting how the .NET Core web application handles web requests.

CVE-2018-8269: Denial of Service Vulnerability in OData

Microsoft is releasing this security advisory to provide information about a vulnerability in ASP.NET Core. This advisory also provides guidance on what developers can do to update their applications to remove this vulnerability.

Microsoft is aware of a denial of service attack in the Microsoft OData library used in ASP.NET could cause a denial of service against an OData web application. An unauthenticated, remote attacker could exploit this vulnerability by issuing specially crafted requests to the OData application.

The update addresses the vulnerability by updating the version of OData ASP.NET Core uses.

Getting the Update

The latest .NET Core updates are available on the .NET Core download page. This update is also included in the Visual Studio 15.9.16, 16.0.8 and 16.2.5 which are also releasing today. Choose Check for Updates in the Help menu.

See the .NET Core release notes ( 2.1.13 | 2.2.7 ) for details on the release, including issues fixed and affected packages.

Docker Images

.NET Docker images have been updated for today’s release. The following repos have been updated.

microsoft/dotnet
microsoft/dotnet-samples
microsoft/aspnetcore

Note: Look at the “Tags” view in each repository to see the updated Docker image tags.

Note: You must re-pull base images to get updates. The Docker client does not pull updates automatically.

Azure App Services deployment

Deployment of these updates Azure App Services has been scheduled, and they estimate completion by September 23, 2019.

The post .NET Core September 2019 Updates – 2.1.13 and 2.2.7 appeared first on .NET Blog.


.NET Framework September 2019 Security and Quality Rollup

$
0
0

Today, we are releasing the September 2019 Cumulative Update, Security and Quality Rollup, and Security Only Update for .NET Framework.

Security

CVE-2019-1142– .NET Framework Elevation of Privilege Vulnerability

An elevation of privilege vulnerability exists when the .NET Framework common language runtime (CLR) allows file creation in arbitrary locations. An attacker who successfully exploited this vulnerability could write files to folders that require higher privileges than what the attacker already has.

To exploit the vulnerability, an attacker would need to log into a system. The attacker could then specify the targeted folder and trigger an affected process to run.

This update addresses the vulnerability correcting how the .NET Framework CLR process logs data.

CVE-2019-1142

 

Getting the Update

The Cumulative Update and Security and Quality Rollup are available via Windows Update, Windows Server Update Services, Microsoft Update Catalog, and Docker.  The Security Only Update is available via Windows Server Update Services and Microsoft Update Catalog.

 

Microsoft Update Catalog

You can get the update via the Microsoft Update Catalog. For Windows 10, NET Framework 4.8 updates are available via Windows Update, Windows Server Update Services, Microsoft Update Catalog.  Updates for other versions of .NET Framework are part of the Windows 10 Monthly Cumulative Update.

Note: Customers that rely on Windows Update and Windows Server Update Services will automatically receive the .NET Framework version-specific updates. Advanced system administrators can also take use of the below direct Microsoft Update Catalog download links to .NET Framework-specific updates. Before applying these updates, please ensure that you carefully review the .NET Framework version applicability, to ensure that you only install updates on systems where they apply.

The following table is for Windows 10 and Windows Server 2016+ versions.

Product Version Cumulative Update
Windows 10 1903 (May 2019 Update)                
.NET Framework 3.5, 4.8 Catalog
4514359
Windows 10 1809 (October 2018 Update)
Windows Server 2019

4514601
 
.NET Framework 3.5, 4.7.2 Catalog
4514366
.NET Framework 3.5, 4.8 Catalog
4514358
Windows 10 1803 (April 2018 Update)  
.NET Framework 3.5, 4.7.2 Catalog
4516058
.NET Framework 4.8 Catalog
4514357
Windows 10 1709 (Fall Creators Update)               
.NET Framework 3.5, 4.7.1, 4.7.2 Catalog
4516066
.NET Framework 4.8 Catalog
4514356
Windows 10 1703 (Creators Update)  

 

.NET Framework 3.5, 4.7, 4.7.1, 4.7.2 Catalog
4516068
.NET Framework 4.8 Catalog
4514355
Windows 10 1607 (Anniversary Update)
Windows Server 2016
 
.NET Framework 3.5, 4.6.2, 4.7, 4.7.1, 4.7.2 Catalog
4516044
.NET Framework 4.8 Catalog
4514354
Windows 10 1507  
.NET Framework 3.5, 4.6, 4.6.1, 4.6.2 Catalog
4516070

 

The following table is for earlier Windows and Windows Server versions.

Product Version Security and Quality Rollup Security Only Update
Windows 8.1
Windows RT 8.1
Windows Server 2012 R2

Catalog
4514604
 

Catalog
4514599
 
.NET Framework 3.5 Catalog
4514371
Catalog
4514350
.NET Framework 4.5.2 Catalog
4514367
Catalog
4514341
.NET Framework 4.6, 4.6.1, 4.6.2, 4.7, 4.7.1, 4.7.2 Catalog
4514364
Catalog
4514338
.NET Framework 4.8 Catalog
4514361
Catalog
4514331
 

Windows Server 2012

 

 

Catalog
4514603

 

 

Catalog
4514598

 

.NET Framework 3.5 Catalog
4514370
Catalog
4514349
.NET Framework 4.5.2 Catalog
4514368
Catalog
4514342
.NET Framework 4.6, 4.6.1, 4.6.2, 4.7, 4.7.1, 4.7.2 Catalog
4514363
Catalog
4514337
.NET Framework 4.8 Catalog
4514360
Catalog
4514330
 

Windows 7 SP1
Windows Server 2008 R2 SP1

 


Catalog
4514602
 
 

N/A

 

.NET Framework 3.5.1 Catalog
4507004
N/A
.NET Framework 4.5.2 Catalog
4507001
N/A
.NET Framework 4.6, 4.6.1, 4.6.2, 4.7, 4.7.1, 4.7.2 Catalog
4511516
N/A
.NET Framework 4.8 Catalog
4511525
N/A
 

Windows Server 2008

 


Catalog
4514605
 
 

N/A

 

.NET Framework 2.0, 3.0 Catalog
4507003
N/A
.NET Framework 4.5.2 Catalog
4507001
N/A
.NET Framework 4.6 Catalog
4511516
N/A

Docker Images

We will be updating the following .NET Framework container images later today:

Note: You must re-pull base images in order to get updates. The Docker client does not pull updates automatically.

Previous Monthly Rollups

The last few .NET Framework Monthly updates are listed below for your convenience:

The post .NET Framework September 2019 Security and Quality Rollup appeared first on .NET Blog.

Finalization implementation details

$
0
0

Years ago I wrote a document on making finalization scanning concurrent. At the time there was an internal team that was using finalization as a way to resurrect objects and putting them back in their cache. While we’ve always advised to folks that finalization was for releasing native resources I couldn’t fault this team for using it the way they did. And of course finalization scanning was taking quite some time because well, there were so many finalizable objects to scan so I wanted to make this part concurrent. As part of the on-going latency reduction effort we’ve finally put this on the agenda. Of course between the time I wrote that doc and now, things have changed quite a bit and there are new considerations I have to make so I’m still thinking about the design. While I’m thinking about it I thought it would be interesting to explain some of the finalization implementation details so that’s what I’ll do in this blog entry. I’m in the process of working on a design doc for concurrent finalization scanning that I will be checking into the coreclr repo.

First of all I wanted to clarify some terminology. I’ve seen some confusing terms in various articles and books. So instead of using something that’s interpreted differently by different people I will just use the terms used in the GC code.

Internally we have a CFinalize class that manages the finalization. The only consumers of this class are the GC itself and of course the finalizer thread. And there’s only one finalizer thread (over the years there were talks about having multiple finalizer threads but there wasn’t enough justification; I have heard that there’s usage that actually depends on the fact there’s only one finalizer thread. While we have no plans to make this multiple I would strong suggest against doing that). The finalizer thread runs at THREAD_PRIORITY_HIGHEST. And this is an important detail as it will have performance implications that I will mention below.

The CFinalize class has an array which we use to track all finalizable objects. And we divide this array into a few parts that I will call segs (as in segments, not to be confused with the GC heap segments). The array looks like this – from lower addresses to higher addresses:

Gen2 seg | Gen1 seg | Gen0 seg | Critical Finalizer seg | Finalizer seg | Free seg

(I’m just calling each portion a “seg” so it’s easier to refer to them)

The genX segs are to record finalizable objects that are still live, ie, either in the generation that GC hasn’t collected so all objects in those generations are live by definition, or is held live by user code. When GC discovers an object to be dead, it then promotes the object which means the object is now tracked by either the Critical Finalizer seg or the Finalizer seg which I will refer to as CF/F segs (this is what’s usually referred to as f-reachable).

When the finalizer thread actually runs finalizers they would pick them off from the CF/F segs. And the ones whose finalizers have been run are now tracked in the Free seg (although saying tracked can be misleading as currently we just treat this seg as “we no longer care about the objects in this seg” and the seg is really just to indicate the free slots we have in the array).

Within GC, each heap has its own CFinalize instance which is called finalize_queue. For lifetime tracking, GC does the following with finalize_queue:

  • The CF/F segs are considered as a source of roots so they will be scanned to indicate which objects should be live. This is what CFinalize::GcScanRoots does. Because the finalizer thread runs at high priority, it’s very likely that there’s nothing here to scan because when we were done with the GC that discovered these objects and moved them to these segs, the finalizer thread was allowed to run and would’ve quickly finished running the finalizers (unless of course the finalizers were blocked for some reason – so there you go, another reason why it’s bad to have your finalizers block which means GC will need to scan the CF/F segs again).

Of course there are other sources of roots like the stack or GC handles. After we are done marking all the objects held live, directly or indirectly by all those roots, we now have the full knowledge of object lifetime.

  • Then we scan the genX segs to see which objects tracked there are dead, in other words, not promoted (!IsPromoted(obj)), and for those objects we need to promote them so the finalizers can run. So these newly promoted objects are promoted due to finalization. This is what CFinalize::ScanForFinalization does. There are exceptions to this – if it’s a WeakReference or WeakReference<T>, we just run what the finalizer is supposed to do (remember finalizers are written in managed code and run on the finalizer thread – we wouldn’t be running the finalizer the normal way when the EE is suspended) and *not* promote those objects ’cause now we have no reason to. This is a nice perf shortcut. The other exception is when the finalizer is suppressed.

There’s another relevant thing to mention which is the short and long weak handles. These are the handle types we use in the runtime. They are exposed as GCHandleType.Weak and GCHandleType.WeakTrackResurrection respectively. Before we do ScanForFinalization, we null the target of short weak handles if the target object is not promoted. After ScanForFinalization, we do that for long weak handles. So what distinguishes these 2 handle types is the promotion due to finalization.

The post Finalization implementation details appeared first on .NET Blog.

Announcing .NET Core 3.0 Release Candidate 1

$
0
0

Today, we’re announcing .NET Core 3.0 Release Candidate 1. Just like with Preview 9, we’ve focused on polishing .NET Core 3.0 for a final release. We are now getting very, very close. We intend to release the final version on September 23 at .NET Conf.

Download .NET Core 3.0 RC1 on Windows, macOS, and Linux, available now.

Details:

Why RC1?

The .NET Core 3.0 Preview 9 post stated that Preview 9 would be the last release before the final GA one. We, or at least the tireless writer of these blog posts, were mistaken. And now for the explanation.

For technical and historical reasons, the .NET toolset (compilers, NuGet client, MSBuild, …) is duplicated between Visual Studio and the .NET Core SDK. Important changes were made in the toolset as part of Visual Studio 2019 16.3 Preview 4, also released today. It is critical that the .NET Core SDK version that is part of any Visual Studio release includes the same toolset in order to deliver a compatible experience in all scenarios.

We should have realized that there was a high likelihood that we might need to release changes to accomodate another Visual Studio preview. Making fixes in the .NET toolset like this is standard operating procedure. We could have released a new .NET Core SDK and only delivered it via Visual Studio, however, we’ve broken people in the (now distant) past with that approach. As a result, when we release a new .NET Core SDK, we make it available for everyone in all the places.

Visual Studio Support

.NET Core 3.0 is supported with Visual Studio 2019 16.3 Preview 4 and Visual Studio for Mac 8.3, which were also released today. Please upgrade to it for the best (and supported) experience with .NET Core 3.0 Preview RC1. See Visual Studio 2019 16.3 release notes for more information.

The C# Extension for Visual Studio Code is always updated to support new .NET Core versions. Make sure you have the latest version of the C# extension installed.

Go Live

NET Core 3.0 Preview RC1 is supported by Microsoft and can be used in production. We strongly recommend that you test your app running on Preview RC1 before deploying into production. If you find an issue with .NET Core 3.0, please file a GitHub issue and/or contact Microsoft support.

Closing

The .NET Core 3.0 release is coming close to completion, and the team is solely focused on stability and reliability now that we’re no longer building new features. Please tell us about any issues you find, ideally as quickly as possible. We want to get as many fixes in as possible before we ship the final 3.0 release.

The post Announcing .NET Core 3.0 Release Candidate 1 appeared first on .NET Blog.

Release Candidate builds of Entity Framework Core 3.0 and Entity Framework 6.3 are now available

$
0
0

We previously said that preview 9 would be your last chance to test EF Core 3.0 and EF 6.3 before general availability. But it turns out that we made enough improvements to our libraries and across the whole of .NET Core 3.0 to justify publishing a release candidate build. Hence the packages for EF Core 3.0 RC1 and EF 6.3 RC1 were uploaded to nuget.org today.

Consider installing daily builds

Although RC1 builds contain several improvements, we took several more critical bug fixes after the RC1 branch was created.

Detailed instructions to install daily builds, including the necessary NuGet feeds, can be found in the How to get daily builds of ASP.NET Core article.

For the best experience with daily builds, install at least the 3.0 RC1 version of the .NET Core SDK, and ASP.NET Core, which where also published today.

Here are a couple of the most relevant improvements you may want to verify:

  • Work on the EF Core in-memory provider was finished and most query features should now be working (the majority of it went into RC1)
  • EF Core’s compilation performance was improved significantly for complex queries

For other details on recent changes and how to install the packages, breaking changes and known workarounds, please refer to the information in the preview 9 blog post.

What happens next

We intend to release the final versions of EF Core 3.0 and EF 6.3 on September 23 at .NET Conf, so we are very close to done.

The team has switched focus to to updating our documentation for the new releases, and to the 3.1 release, which we plan to release later this year.

Thank you

Thank you for helping make this a better release, and please keep the feedback coming! Although any important bugs reported at this stage will likely not make it into 3.0, we will consider them for 3.1.

The post Release Candidate builds of Entity Framework Core 3.0 and Entity Framework 6.3 are now available appeared first on .NET Blog.

Announcing .NET Core 3.0

$
0
0

Announcing .NET Core 3.0

We’re excited to announce the release of .NET Core 3.0. It includes many improvements, including adding Windows Forms and WPF, adding new JSON APIs, support for ARM64 and improving performance across the board. C# 8 is also part of this release, which includes nullable, async streams, and more patterns. F# 4.7 is included, and focused on relaxing syntax and targeting .NET Standard 2.0. You can start updating existing projects to target .NET Core 3.0 today. The release is compatible with previous versions, making updating easy.

Watch the team and the community talk about .NET at .NET Conf, live NOW!

You can download .NET Core 3.0, for Windows, macOS, and Linux:

ASP.NET Core 3.0 and EF Core 3.0 are also releasing today.

Visual Studio 2019 16.3 and Visual Studio for Mac 8.3 were also released today and are required update to use .NET Core 3.0 with Visual Studio. .NET Core 3.0 is part of Visual Studio 2019 16.3. You can just get .NET Core by simply upgrading Visual Studio 2019 16.3.

Thank you to everyone that contributed to .NET Core 3.0! Hundreds of people were involved in making this release happen, including major contributions from the community.

Release notes:

Note: There are some contributors missing from the contributor list. We’re working on fixing that. Send mail to dotnet@microsoft.com if you are missing.

What you should know about 3.0

There are some key improvements and guidance that are important to draw attention to before we go into a deep dive on all the new features in .NET Core 3.0. Here’s the quick punch list.

  • .NET Core 3.0 is already battle-tested by being hosted for months at dot.net and on Bing.com. Many other Microsoft teams will soon be deploying large workloads on .NET Core 3.0 in production.
  • Performance is greatly improved across many components and is described in detail at Performance Improvements in .NET Core 3.0.
  • C# 8 add async streams, range/index, more patterns, and nullable reference types. Nullable enables you to directly target the flaws in code that lead to NullReferenceException. The lowest layer of the framework libraries has been annotated, so that you know when to expect null.
  • F# 4.7 focuses on making some thing easier with implicit yield expressions and some syntax relaxations. It also includes support for LangVersion, and ships with nameof and opening of static classes in preview. The F# Core Library now also targets .NET Standard 2.0. You can read more at Announcing F# 4.7.
  • .NET Standard 2.1 increases the set of types you can use in code that can be used with both .NET Core and Xamarin. .NET Standard 2.1 includes types since .NET Core 2.1.
  • Windows Desktop apps are now supported with .NET Core, for both Windows Forms and WPF (and open source). The WPF designer is part of Visual Studio 2019 16.3. The Windows Forms designer is still in preview and available as a VSIX download.
  • .NET Core apps now have executables by default. In past releases, apps needed to be launched via the dotnet command, like dotnet myapp.dll. Apps can now be launched with an app-specific executable, like myapp or ./myapp, depending on the operating system.
  • High performance JSON APIs have been added, for reader/writer, object model and serialization scenarios. These APIs were built from scratch on top of Span<T> and use UTF8 under the covers instead of UTF16 (like string). These APIs minimize allocations, resulting in faster performance, and much less work for the garbage collector. See The future of JSON in .NET Core 3.0.
  • The garbage collector uses less memory by default, often a lot less. This improvement is very beneficial for scenarios where many applications are hosted on the same server. The garbage collector has also been updated to make better use of large numbers of cores, on machines with >64 cores.
  • .NET Core has been hardened for Docker to enable .NET applications to work predictably and efficiently in containers. The garbage collector and thread pool have been updated to work much better when a container has been configured for limited memory or CPU. .NET Core docker images are smaller, particularly the SDK image.
  • Raspberry Pi and ARM chips are now supported to enable IoT development, including with the remote Visual Studio debugger. You can deploy apps that listen to sensors, and print messages or images on a display, all using the new GPIO APIs. ASP.NET can be used to expose data as an API or as a site that enables configuring an IoT device.
  • .NET Core 3.0 is a ‘current’ release and will be superseded by .NET Core 3.1, targeted for November 2019. .NET Core 3.1 will be a long-term supported (LTS) release (supported for at least 3 years). We recommend that you adopt .NET Core 3.0 and then adopt 3.1. It’ll be very easy to upgrade.
  • .NET Core 2.2 will go EOL on 12/23 as it is now the previous ‘current’ release. See .NET Core support policy.
  • .NET Core 3.0 will be available with RHEL 8 in the Red Hat Application Streams, after several years of collaboration with Red Hat.
  • Visual Studio 2019 16.3 is a required update for Visual Studio users on Windows that want to use .NET Core 3.0.
  • Visual Studio for Mac 8.3 is a required update for Visual Studio for Mac users that want to use .NET Core 3.0.
  • Visual Studio Code users should just always use the latest version of the C# extension to ensure that the newest scenarios work, including targeting .NET Core 3.0.
  • Azure App Service deployment of .NET Core 3.0 is currently ongoing.
  • Azure Dev Ops deployment of .NET Core 3.0 is coming soon. Will update when it is available.

Platform support

.NET Core 3.0 is supported on the following operating systems:

  • Alpine: 3.9+
  • Debian: 9+
  • openSUSE: 42.3+
  • Fedora: 26+
  • Ubuntu: 16.04+
  • RHEL: 6+
  • SLES: 12+
  • macOS: 10.13+
  • Windows Client: 7, 8.1, 10 (1607+)
  • Windows Server: 2012 R2 SP1+

Note: Windows Forms and WPF apps only work on Windows.

Chip support follows:

  • x64 on Windows, macOS, and Linux
  • x86 on Windows
  • ARM32 on Windows and Linux
  • ARM64 on Linux (kernel 4.14+)

Note: Please ensure that .NET Core 3.0 ARM64 deployments use Linux kernel 4.14 version or later. For example, Ubuntu 18.04 satisfies this requirement, but 16.04 does not.

WPF and Windows Forms

You can build WPF and Windows Forms apps with .NET Core 3, on Windows. We’ve had a strong compatibility goal from the start of the project, to make it easy to migrate desktop applications from .NET Framework to .NET Core. We’ve heard feedback from many developers that have already successfully ported their app to .NET Core 3.0 that the process is straightforward. To a large degree, we took WPF and Windows Forms as-is, and got them working on .NET Core. The engineering project was very different than that, but that’s a good way to think about the project.

The following image shows a .NET Core Windows Forms app:

Visual Studio 2019 16.3 has support for creating WPF apps that target .NET Core. This includes new templates and an updated XAML designer and XAML Hot Reload. The designer is similar to the existing XAML designer (that targets .NET Framework), however, you may notice some differences in experience. The big technical difference is that the designer for .NET Core uses a new surface process (wpfsurface.exe) to solely run the runtime code targeting the .NET Core version. Previously, the .NET Framework WPF designer process (xdesproc.exe) was a itself a WPF .NET Framework process hosting the designer, and due to runtime incompatibility we can’t have a WPF .NET Framework process (in this case, Visual Studio) loading two versions of .NET (.NET Framework and .NET Core) into the same process. This means that some aspects of the designer, like designer extensions, can’t work in the same way. If you are writing designer extensions, we recommend reading XAML designer extensibility migration.

The following image shows a WPF app being displayed in the new designer:

The Windows Forms designer is still in preview, and available as a separate download. It will be added to Visual Studio as part of a later release. The designer currently includes support for the most commonly used controls and low-level functionality. We’ll keep improving the designer with monthly updates. We don’t recommend porting your Windows Forms applications to .NET Core just yet, particularly if you rely on the designer. Please do experiment with the designer preview, and give us feedback.

You can also create and build desktop applications from the command line using the .NET CLI.

For example, you can quickly create a new Windows Forms app:

dotnet new winforms -o myapp
cd myapp
dotnet run

You can try WPF using the same flow:

dotnet new wpf -o mywpfapp
cd mywpfapp
dotnet run

We made Windows Forms and WPF open source, back in December 2018. It’s been great to see the community and the Windows Forms and WPF teams working together to improve those UI frameworks. In the case of WPF, we started out with a very small amount of code in the GitHub repo. At this point, almost all of WPF has been published to GitHub, and a few more components will straggle in over time. Like other .NET Core projects, these new repos are part of the .NET Foundation and licensed with the MIT license.

The System.Windows.Forms.DataVisualization package (which includes the chart control) is also available for .NET Core. You can now include this control in your .NET Core WinForms applications. The source for the chart control is available at dotnet/winforms-datavisualization, on GitHub. The control was migrated to ease porting to .NET Core 3, but isn’t a component we expect to update significantly.

Windows Native Interop

Windows offers a rich native API, in the form of flat C APIs, COM and WinRT. We’ve had support for P/Invoke since .NET Core 1.0, and have been adding the ability to CoCreate COM APIs, activate WinRT APIs, and exposed managed code as COM components as part of the .NET Core 3.0 release. We have had many requests for these capabilities, so we know that they will get a lot of use.

Late last year, we announced that we had managed to automate Excel from .NET Core. That was a fun moment. Under the covers, this demo is using COM interop features like NOPIA, object equivalence and custom marshallers. You can now try this and other demos yourself at extension samples.

Managed C++ and WinRT interop have partial support with .NET Core 3.0 and will be included with .NET Core 3.1.

Nullable reference types

C# 8.0 introduces nullable reference types and non-nullable reference types that enable you to make important statements about the properties for reference type variables:

  • A reference is not supposed to be null. When variables aren’t supposed to be null, the compiler enforces rules that ensure it is safe to dereference these variables without first checking that it isn’t null.
  • A reference may be null. When variables may be null, the compiler enforces different rules to ensure that you’ve correctly checked for a null reference.

This new feature provides significant benefits over the handling of reference variables in earlier versions of C# where the design intent couldn’t be determined from the variable declaration. With the addition of nullable reference types, you can declare your intent more clearly, and the compiler both helps you do that correctly and discover bugs in your code.

See This is how you get rid of null reference exceptions forever, Try out Nullable Reference Types and Nullable reference types to learn more.

Default implementations of interface members

Today, once you publish an interface, it’s game over for changing it: you can’t add members to it without breaking all the existing implementers of it.

With C# 8.0, you can provide a body for an interface member. As a result, if a class that implements the interface doesn’t implement that member (perhaps because it wasn’t there yet when they wrote the code), then the calling code will just get the default implementation instead.

interface ILogger
{
    void Log(LogLevel level, string message);
    void Log(Exception ex) => Log(LogLevel.Error, ex.ToString()); // New overload
}
class ConsoleLogger : ILogger
{
    public void Log(LogLevel level, string message) { ... }
    // Log(Exception) gets default implementation
}

In this example, the ConsoleLogger class doesn’t have to implement the Log(Exception) overload of ILogger, because it is declared with a default implementation. Now you can add new members to existing public interfaces as long as you provide a default implementation for existing implementors to use.

Async streams

You can now foreach over an async stream of data using IAsyncEnumerable<T>. This new interface is exactly what you’d expect; an asynchronous version of IEnumerable<T>. The language lets you await foreach over tasks to consume their elements. On the production side, you yield return items to produce an async stream. It might sound a bit complicated, but it is incredibly easy in practice.

The following example demonstrates both production and consumption of async streams. The foreach statement is async and itself uses yield return to produce an async stream for callers. This pattern – using yield return — is the recommended model for producing async streams.

async IAsyncEnumerable<int> GetBigResultsAsync()
{
    await foreach (var result in GetResultsAsync())
    {
        if (result > 20) yield return result;
    }
}

In addition to being able to await foreach, you can also create async iterators, e.g. an iterator that returns an IAsyncEnumerable/IAsyncEnumerator that you can both await and yield return in. For objects that need to be disposed, you can use IAsyncDisposable, which various framework types implement, such as Stream and Timer.

Index and Range

We’ve created new syntax and types that you can use to describe indexers, for array element access or for any other type that exposes direct data access. This includes support for both a single value — the usual definition of an index — or two values, which describing a range.

Index is a new type that describes an array index. You can create an Index from an int that counts from the beginning, or with a prefix ^ operator that counts from the end. You can see both cases in the following example:

Index i1 = 3;  // number 3 from beginning
Index i2 = ^4; // number 4 from end
int[] a = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 };
Console.WriteLine($"{a[i1]}, {a[i2]}"); // "3, 6"

Range is similar, consisting of two Index values, one for the start and one for the end, and can be written with a x..y range expression. You can then index with a Range in order to produce a slice of the underlying data, as demonstrated in the following example:

var slice = a[i1..i2]; // { 3, 4, 5 }

Using Declarations

Are you tired of using statements that require indenting your code? No more! You can now write the following code, which attaches a using declaration to the scope of the current statement block and then disposes the object at the end of it.

using System;
using System.Linq;
using System.Collections.Generic;
using static System.Console;
using System.IO;

namespace usingapp
{
    class Program
    {
        static void Main()
        {
            var filename = "Program.cs";
            var line = string.Empty;
            var magicString = "magicString";

            var file = new FileInfo(filename);
            using var reader = file.OpenText();
            while ((line = reader.ReadLine())!= null)
            {
                if (line.Contains(magicString))  
                { 
                    WriteLine("Found string"); 
                    return;
                }
            }

            WriteLine("String not found");
        } // reader disposed here
    }
}

Switch Expressions

Anyone who uses C# probably loves the idea of a switch statement, but not the syntax. C# 8 introduces switch expressions, which enable the following:

  • terser syntax
  • returns a value since it is an expression
  • fully integrated with pattern matching

The switch keyword is “infix”, meaning the keyword sits between the tested value (that’s o in the first example) and the list of cases, much like expression lambdas.

The first examples uses the lambda syntax for methods, which integrates well with the switch expressions but isn’t required.

static string Display(object o) => o switch
{
    Point { X: 0, Y: 0 }         => "origin",
    Point { X: var x, Y: var y } => $"({x}, {y})",
    _                            => "unknown"
};

There are two patterns at play in this example. o first matches with the Point type pattern and then with the property pattern inside the {curly braces}. The _ describes the discard pattern, which is the same as default for switch statements.

You can go one step further, and rely on tuple deconstruction and parameter position, as you can see in the following example:

static State ChangeState(State current, Transition transition, bool hasKey) =>
    (current, transition) switch
    {
        (Opened, Close)              => Closed,
        (Closed, Open)               => Opened,
        (Closed, Lock)   when hasKey => Locked,
        (Locked, Unlock) when hasKey => Closed,
        _ => throw new InvalidOperationException($"Invalid transition")
    };

In this example, you can see you do not need to define a variable or explicit type for each of the cases. Instead, the compiler can match the tuple being testing with the tuples defined for each of the cases.

All of these patterns enable you to write declarative code that captures your intent instead of procedural code that implements tests for it. The compiler becomes responsible for implementing that boring procedural code and is guaranteed to always do it correctly.

There will still be cases where switch statements will be a better choice than switch expressions and patterns can be used with both syntax styles.

Introducing a fast JSON API

.NET Core 3.0 includes a new family of JSON APIs that enable reader/writer scenarios, random access with a document object model (DOM) and a serializer. You are likely familiar with using Json.NET. The new APIs are intended to satisfy many of the same scenarios, but with less memory and faster execution.

You can see the initial motivation and description of the plan in The future of JSON in .NET Core 3.0. This includes James Netwon-King, the author of Json.Net, explaining why a new API was created, as opposed to extending Json.NET. In short, we wanted to build a new JSON API that took advantage of all the new performance capabilities in .NET Core, and delivered performance inline with that. It wasn’t possible to do that in an existing code-base like Json.NET while maintaining compatibility.

Let’s take a quick look at the new API, layer by layer.

Utf8JsonReader

System.Text.Json.Utf8JsonReader is a high-performance, low allocation, forward-only reader for UTF-8 encoded JSON text, read from a ReadOnlySpan<byte>. The Utf8JsonReader is a foundational, low-level type, that can be leveraged to build custom parsers and deserializers. Reading through a JSON payload using the new Utf8JsonReader is 2x faster than using the reader from Json.NET. It does not allocate until you need to actualize JSON tokens as (UTF16) strings.

Utf8JsonWriter

System.Text.Json.Utf8JsonWriter provides a high-performance, non-cached, forward-only way to write UTF-8 encoded JSON text from common .NET types like String, Int32, and DateTime. Like the reader, the writer is a foundational, low-level type, that can be leveraged to build custom serializers. Writing a JSON payload using the new Utf8JsonWriter is 30-80% faster than using the writer from Json.NET and does not allocate.

JsonDocument

System.Text.Json.JsonDocument provides the ability to parse JSON data and build a read-only Document Object Model (DOM) that can be queried to support random access and enumeration. It is built on top of the Utf8JsonReader. The JSON elements that compose the data can be accessed via the JsonElement type which is exposed by the JsonDocument as a property called RootElement. The JsonElement contains the JSON array and object enumerators along with APIs to convert JSON text to common .NET types. Parsing a typical JSON payload and accessing all its members using the JsonDocument is 2-3x faster than Json.NET with very little allocations for data that is reasonably sized (i.e. < 1 MB).

JSON Serializer

System.Text.Json.JsonSerializer layers on top of the high-performance Utf8JsonReader and Utf8JsonWriter. It deserializes objects from JSON and serializes objects to JSON. Memory allocations are kept minimal and includes support for reading and writing JSON with Stream asynchronously.

See the documentation for information and samples.

Introducing the new SqlClient

SqlClient is the data provider you use to access Microsoft SQL Server and Azure SQL Database, either through one of the popular .NET O/RMs, like EF Core or Dapper, or directly using the ADO.NET APIs. It will now be released and updated as the Microsoft.Data.SqlClient NuGet package, and supported for both .NET Framework and .NET Core applications. By using NuGet, it will be easier for the SQL team to provide updates to both .NET Framework and .NET Core users.

ARM and IoT Support

We added support for Linux ARM64 this release, after having added support for ARM32 for Linux and Windows in the .NET Core 2.1 and 2.2, respectively. While some IoT workloads take advantage of our existing x64 capabilities, many users had been asking for ARM support. That is now in place, and we are working with customers who are planning large deployments.

Many IoT deployments using .NET are edge devices, and entirely network-oriented. Other scenarios require direct access to hardware. In this release, we added the capability to use serial ports on Linux and take advantage of digital pins on devices like the Raspberry Pi. The pins use a variety of protocols. We added support for GPIO, PWM, I2C, and SPI, to enable reading sensor data, interacting with radios and writing text and images to displays, and many other scenarios.

This functionality is available as part of the following packages:

As part of providing support for GPIO (and friends), we took a look at what was already available. We found APIs for C# and also Python. In both cases, the APIs were wrappers over native libraries, which were often licensed as GPL. We didn’t see a path forward with that approach. Instead, we built a 100% C# solution to implement these protocols. This means that our APIs will work anywhere .NET Core is supported, can be debugged with a C# debugger (via sourcelink), and supports multiple underlying Linux drivers (sysfs, libgpiod, and board-specific). All of the code is licensed as MIT. We see this approach as a major improvement for .NET developers compared to what has existed.

See dotnet/iot to learn more. The best places to start are samples or devices. We have built a few experiments while adding support for GPIO. One of them was validating that we could control an Arduino from a Pi through a serial port connection. That was suprisingly easy. We also spent a lot of time playing with LED matrices, as you can see in this RGB LED Matrix sample. We expect to share more of these experiments over time.

.NET Core runtime roll-forward policy update

The .NET Core runtime, actually the runtime binder, now enables major-version roll-forward as an opt-in policy. The runtime binder already enables roll-forward on patch and minor versions as a default policy. We decided to expose a broader set of policies, which we expected would be important for various scenarios, but did not change the default roll-forward behavior.

There is a new property called RollForward, which accepts the following values:

  • LatestPatch — Rolls forward to the highest patch version. This disables the Minor policy.
  • Minor — Rolls forward to the lowest higher minor version, if the requested minor version is missing. If the requested minor version is present, then the LatestPatch policy is used. This is the default policy.
  • Major — Rolls forward to lowest higher major version, and lowest minor version, if the requested major version is missing. If the requested major version is present, then the Minor policy is used.
  • LatestMinor — Rolls forward to highest minor version, even if the requested minor version is present.
  • LatestMajor — Rolls forward to highest major and highest minor version, even if requested major is present.
  • Disable — Do not roll forward. Only bind to specified version. This policy is not recommended for general use since it disable the ability to roll-forward to the latest patches. It is only recommended for testing.

See Runtime Binding Behavior and dotnet/core-setup #5691 for more information.

Docker and cgroup Limits

Many developers are packaging and running their application with containers. A key scenario is limiting a container’s resources such as CPU or memory. We implemented support for memory limits back in 2017. Unfortunately, we found that the implementation wasn’t aggressive enough to reliably stay under the configured limits and applications were still being OOM killed when memory limits are set (particular <500MB). We have fixed that in .NET Core 3.0. We strongly recommend that .NET Core Docker users upgrade to .NET Core 3.0 due to this improvement.

The Docker resource limits feature is built on top of cgroups, which a Linux kernel feature. From a runtime perspective, we need to target cgroup primitives.

You can limit the available memory for a container with the docker run -m argument, as shown in the following example that creates an Alpine-based container with a 4MB memory limit (and then prints the memory limit):

C:\>docker run -m 4mb --rm alpine cat /sys/fs/cgroup/memory/memory.limit_in_bytes
4194304

We also added made changes to better support CPU limits (--cpus). This includes changing the way that the runtime rounds up or down for decimal CPU values. In the case where --cpus is set to a value close (enough) to a smaller integer (for example, 1.499999999), the runtime would previously round that value down (in this case, to 1). As a result, the runtime would take advantage of less CPUs than requested, leading to CPU underutilization. By rounding up the value, the runtime augments the pressure on the OS threads scheduler, but even in the worst case scenario (--cpus=1.000000001 — previously rounded down to 1, now rounded to 2), we have not observed any overutilization of the CPU leading to performance degradation.

The next step was ensuring that the thread pool honors CPU limits. Part of the algorithm of the thread pool is computing CPU busy time, which is, in part, a function of available CPUs. By taking CPU limits into account when computing CPU busy time, we avoid various heuristics of the threadpool competing with each other: one trying to allocate more threads to increase the CPU busy time, and the other one trying to allocate less threads because adding more threads doesn’t improve the throughput.

Making GC Heap Sizes Smaller by default

While working on improving support for docker memory limits, we were inspired to make more general GC policy updates to improve memory usage for a broader set of applications (even when not running in a container). The changes better align the generation 0 allocation budget with modern processor cache sizes and cache hierarchy.

Damian Edwards on our team noticed that the memory usage of the ASP.NET benchmarks were cut in half with no negative effect on other performance metrics. That’s a staggering improvement! As he says, these are the new defaults, with no change required to his (or your) code (other than adopting .NET Core 3.0).

The memory savings that we saw with the ASP.NET benchmarks may or may not be representative of what you’ll see with your application. We’d like to hear how these changes reduce memory usage for your application.

Better support for many proc machines

Based on .NET’s Windows heritage, the GC needed to implement the Windows concept of processor groups to support machines with 64+ processors. This implementation was made in .NET Framework, 5-10 years ago. With .NET Core, we made the choice initially for the Linux PAL to emulate that same concept, even though it doesn’t exist in Linux. We have since abandoned this concept in the GC and transitioned it exclusively to the Windows PAL.

The GC now exposes a configuration switch, GCHeapAffinitizeRanges, to specify affinity masks on machines with 64+ processors. Maoni Stephens wrote about this change in Making CPU configuration better for GC on machines with > 64 CPUs.

GC Large page support

Large Pages or Huge Pages is a feature where the operating system is able to establish memory regions larger than the native page size (often 4K) to improve performance of the application requesting these large pages.

When a virtual-to-physical address translation occurs, a cache called the Translation lookaside buffer (TLB) is first consulted (often in parallel) to check if a physical translation for the virtual address being accessed is available, to avoid doing a potentially expensive page-table walk. Each large-page translation uses a single translation buffer inside the CPU. The size of this buffer is typically three orders of magnitude larger than the native page size; this increases the efficiency of the translation buffer, which can increase performance for frequently accessed memory. This win can be even more significant in a virtual machine, which has a two-layer TLB.

The GC can now be configured with the GCLargePages opt-in feature to choose to allocate large pages on Windows. Using large pages reduces TLB misses therefore can potentially increase application perf in general, however, the feature has its own set of limitations that should be considered. Bing has experimented with this feature and seen performance improvements.

.NET Core Version APIs

We have improved the .NET Core version APIs in .NET Core 3.0. They now return the version information you would expect. These changes while they are objectively better are technically breaking and may break applications that rely on existing version APIs for various information.

You can now get access to the following version information:

C:\git\testapps\versioninfo>dotnet run
**.NET Core info**
Environment.Version: 3.0.0
RuntimeInformation.FrameworkDescription: .NET Core 3.0.0
CoreCLR Build: 3.0.0
CoreCLR Hash: ac25be694a5385a6a1496db40de932df0689b742
CoreFX Build: 3.0.0
CoreFX Hash: 1bb52e6a3db7f3673a3825f3677b9f27b9af99aa

**Environment info**
Environment.OSVersion: Microsoft Windows NT 6.2.9200.0
RuntimeInformation.OSDescription: Microsoft Windows 10.0.18970
RuntimeInformation.OSArchitecture: X64
Environment.ProcessorCount: 8

Event Pipe improvements

Event Pipe now supports multiple sessions. This means that you can consume events with EventListener in-proc and simultaneously have out-of-process event pipe clients.

New Perf Counters added:

  • % Time in GC
  • Gen 0 Heap Size
  • Gen 1 Heap Size
  • Gen 2 Heap Size
  • LOH Heap Size
  • Allocation Rate
  • Number of assemblies loaded
  • Number of ThreadPool Threads
  • Monitor Lock Contention Rate
  • ThreadPool Work Items Queue
  • ThreadPool Completed Work Items Rate

Profiler attach is now implemented using the same Event Pipe infrastructure.

See Playing with counters from David Fowler to get an idea of what you can do with event pipe to perform your own performance investigations or just monitor application status.

See dotnet-counters to install the dotnet-counters tool.

HTTP/2 Support

We now have support for HTTP/2 in HttpClient. The new protocol is a requirement for some APIs, like gRPC and Apple Push Notification Service. We expect more services to require HTTP/2 in the future. ASP.NET also has support for HTTP/2.

Note: the preferred HTTP protocol version will be negotiated via TLS/ALPN and HTTP/2 will only be used if the server selects to use it.

Tiered Compilation

Tiered compilation was added as an opt-in feature in .NET Core 2.1. It’s a feature that enables the runtime to more adaptively use the Just-In-Time (JIT) compiler to get better performance, both at startup and to maximize throughput. It is enabled by default with .NET Core 3.0. We made a lot of improvements to the feature over the last year, including testing it with a variety of workloads, including websites, PowerShell Core and Windows desktop apps. The performance is a lot better, which is what enabled us to enable it by default.

IEEE Floating-point improvements

Floating point APIs have been updated to comply with IEEE 754-2008 revision. The goal of the .NET Core floating point project is to expose all “required” operations and ensure that they are behaviorally compliant with the IEEE spec.

Parsing and formatting fixes:

  • Correctly parse and round inputs of any length.
  • Correctly parse and format negative zero.
  • Correctly parse Infinity and NaN by performing a case-insensitive check and allowing an optional preceding + where applicable.

New Math APIs:

  • BitIncrement/BitDecrement — corresponds to the nextUp and nextDown IEEE operations. They return the smallest floating-point number that compares greater or lesser than the input (respectively). For example, Math.BitIncrement(0.0) would return double.Epsilon.
  • MaxMagnitude/MinMagnitude — corresponds to the maxNumMag and minNumMag IEEE operations, they return the value that is greater or lesser in magnitude of the two inputs (respectively). For example, Math.MaxMagnitude(2.0, -3.0) would return -3.0.
  • ILogB — corresponds to the logB IEEE operation which returns an integral value, it returns the integral base-2 log of the input parameter. This is effectively the same as floor(log2(x)), but done with minimal rounding error.
  • ScaleB — corresponds to the scaleB IEEE operation which takes an integral value, it returns effectively x * pow(2, n), but is done with minimal rounding error.
  • Log2 — corresponds to the log2 IEEE operation, it returns the base-2 logarithm. It minimizes rounding error.
  • FusedMultiplyAdd — corresponds to the fma IEEE operation, it performs a fused multiply add. That is, it does (x * y) + z as a single operation, there-by minimizing the rounding error. An example would be FusedMultiplyAdd(1e308, 2.0, -1e308) which returns 1e308. The regular (1e308 * 2.0) - 1e308 returns double.PositiveInfinity.
  • CopySign — corresponds to the copySign IEEE operation, it returns the value of x, but with the sign of y.

.NET Platform Dependent Intrinsics

We’ve added APIs that allow access to certain performance-oriented CPU instructions, such as the SIMD or Bit Manipulation instruction sets. These instructions can help achieve big performance improvements in certain scenarios, such as processing data efficiently in parallel. In addition to exposing the APIs for your programs to use, we have begun using these instructions to accelerate the .NET libraries too.

The following CoreCLR PRs demonstrate a few of the intrinsics, either via implementation or use:

For more information, take a look at .NET Platform Dependent Intrinsics, which defines an approach for defining this hardware infrastructure, allowing Microsoft, chip vendors or any other company or individual to define hardware/chip APIs that should be exposed to .NET code.

Supporting TLS 1.3 and OpenSSL 1.1.1 now Supported on Linux

NET Core can now take advantage of TLS 1.3 support in OpenSSL 1.1.1. There are multiple benefits of TLS 1.3, per the OpenSSL team:

  • Improved connection times due to a reduction in the number of round trips required between the client and server
  • Improved security due to the removal of various obsolete and insecure cryptographic algorithms and encryption of more of the connection handshake

.NET Core 3.0 is capable of utilizing OpenSSL 1.1.1, OpenSSL 1.1.0, or OpenSSL 1.0.2 (whatever the best version found is, on a Linux system). When OpenSSL 1.1.1 is available, the SslStream and HttpClient types will use TLS 1.3 when using SslProtocols.None (system default protocols), assuming both the client and server support TLS 1.3.

.NET Core will support TLS 1.3 on Windows and macOS — we expect automatically — when support becomes available.

Cryptography

We added support for AES-GCM and AES-CCM ciphers, implemented via System.Security.Cryptography.AesGcm and System.Security.Cryptography.AesCcm. These algorithms are both Authenticated Encryption with Association Data (AEAD) algorithms, and the first Authenticated Encryption (AE) algorithms added to .NET Core.

NET Core 3.0 now supports the import and export of asymmetric public and private keys from standard formats, without needing to use an X.509 certificate.

All key types (RSA, DSA, ECDsa, ECDiffieHellman) support the X.509 SubjectPublicKeyInfo format for public keys, and the PKCS#8 PrivateKeyInfo and PKCS#8 EncryptedPrivateKeyInfo formats for private keys. RSA additionally supports PKCS#1 RSAPublicKey and PKCS#1 RSAPrivateKey. The export methods all produce DER-encoded binary data, and the import methods expect the same; if a key is stored in the text-friendly PEM format the caller will need to base64-decode the content before calling an import method.

PKCS#8 files can be inspected with the System.Security.Cryptography.Pkcs.Pkcs8PrivateKeyInfo class.

PFX/PKCS#12 files can be inspected and manipulated with System.Security.Cryptography.Pkcs.Pkcs12Info and System.Security.Cryptography.Pkcs.Pkcs12Builder, respectively.

New Japanese Era (Reiwa)

On May 1st, 2019, Japan started a new era called Reiwa. Software that has support for Japanese calendars, like .NET Core, must be updated to accommodate Reiwa. .NET Core and .NET Framework have been updated and correctly handle Japanese date formatting and parsing with the new era.

.NET relies on operating system or other updates to correctly process Reiwa dates. If you or your customers are using Windows, download the latest updates for your Windows version. If running macOS or Linux, download and install ICU version 64.2, which has support the new Japanese era.

Handling a new era in the Japanese calendar in .NET blog has more information about .NET support for the new Japanese era.

Assembly Load Context Improvements

Enhancements to AssemblyLoadContext:

  • Enable naming contexts
  • Added the ability to enumerate ALCs
  • Added the ability to enumerate assemblies within an ALC
  • Made the type concrete – so instantiation is easier (no requirement for custom types for simple scenarios)

See dotnet/corefx #34791 for more details. The appwithalc sample demonstrates these new capabilities.

By using AssemblyDependencyResolver along with a custom AssemblyLoadContext, an application can load plugins so that each plugin’s dependencies are loaded from the correct location, and one plugin’s dependencies will not conflict with another. The AppWithPlugin sample includes plugins that have conflicting dependencies and plugins that rely on satellite assemblies or native libraries.

Assembly Unloadability

Assembly unloadability is a new capability of AssemblyLoadContext. This new feature is largely transparent from an API perspective, exposed with just a few new APIs. It enables a loader context to be unloaded, releasing all memory for instantiated types, static fields and for the assembly itself. An application should be able to load and unload assemblies via this mechanism forever without experiencing a memory leak.

We expect this new capability to be used for the following scenarios:

  • Plugin scenarios where dynamic plugin loading and unloading is required.
  • Dynamically compiling, running and then flushing code. Useful for web sites, scripting engines, etc.
  • Loading assemblies for introspection (like ReflectionOnlyLoad), although MetadataLoadContext will be a better choice in many cases.

Assembly Metadata Reading with MetadataLoadContext

We added MetadataLoadContext, which enables reading assembly metadata without affecting the caller’s application domain. Assemblies are read as data, including assemblies built for different architectures and platforms than the current runtime environment. MetadataLoadContext overlaps with the ReflectionOnlyLoad type, which is only available in the .NET Framework.

MetdataLoadContext is available in the System.Reflection.MetadataLoadContext package. It is a .NET Standard 2.0 package.

Scenarios for MetadataLoadContext include design-time features, build-time tooling, and runtime light-up features that need to inspect a set of assemblies as data and have all file locks and memory freed after inspection is performed.

Native Hosting sample

The team posted a Native Hosting sample. It demonstrates a best practice approach for hosting .NET Core in a native application.

As part of .NET Core 3.0, we now expose general functionality to .NET Core native hosts that was previously only available to .NET Core managed applications through the officially provided .NET Core hosts. The functionality is primarily related to assembly loading. This functionality should make it easier to produce native hosts that can take advantage of the full feature set of .NET Core.

Other API Improvements

We optimized Span<T>, Memory<T> and related types that were introduced in .NET Core 2.1. Common operations such as span construction, slicing, parsing, and formatting now perform better. Additionally, types like String have seen under-the-cover improvements to make them more efficient when used as keys with Dictionary<TKey, TValue> and other collections. No code changes are required to enjoy these improvements.

The following improvements are also new:

  • Brotli support built-in to HttpClient
  • ThreadPool.UnsafeQueueWorkItem(IThreadPoolWorkItem)
  • Unsafe.Unbox
  • CancellationToken.Unregister
  • Complex arithmetic operators
  • Socket APIs for TCP keep alive
  • StringBuilder.GetChunks
  • IPEndPoint parsing
  • RandomNumberGenerator.GetInt32
  • System.Buffers.SequenceReader

Applications now have native executables by default

.NET Core applications are now built with native executables. This is new for framework-dependent application. Until now, only self-contained applications had executables.

You can expect the same things with these executables as you would other native executables, such as:

  • You can double click on the executable to start the application.
  • You can launch the application from a command prompt, using myapp.exe, on Windows, and ./myapp, on Linux and macOS.

The executable that is generated as part of the build will match your operating system and CPU. For example, if you are on a Linux x64 machine, the executable will only work on that kind of machine, not on a Windows machine and not on a Linux ARM machine. That’s because the executables are native code (just like C++). If you want to target another machine type, you need to publish with a runtime argument. You can continue to launch applications with the dotnet command, and not use native executables, if you prefer.

Optimize your .NET Core apps with ReadyToRun images

You can improve the startup time of your .NET Core application by compiling your application assemblies as ReadyToRun (R2R) format. R2R is a form of ahead-of-time (AOT) compilation. It is a publish-time, opt-in feature in .NET Core 3.0.

R2R binaries improve startup performance by reducing the amount of work the JIT needs to do as your application is loading. The binaries contain similar native code as what the JIT would produce, giving the JIT a bit of a vacation when performance matters most (at startup). R2R binaries are larger because they contain both intermediate language (IL) code, which is still needed for some scenarios, and the native version of the same code, to improve startup.

To enable the ReadyToRun compilation:

  • Set the PublishReadyToRun property to true.
  • Publish using an explicit RuntimeIdentifier.

Note: When the application assemblies get compiled, the native code produced is platform and architecture specific (which is why you have to specify a valid RuntimeIdentifier when publishing).

Here’s an example:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.0</TargetFramework>
    <PublishReadyToRun>true</PublishReadyToRun>
  </PropertyGroup>
</Project>

And publish using the following command:

dotnet publish -r win-x64 -c Release

Note: The RuntimeIdentifier can be set to another operating system or chip. It can also be set in the project file.

Assembly linking

The .NET core 3.0 SDK comes with a tool that can reduce the size of apps by analyzing IL and trimming unused assemblies. It is another publish-time opt-in feature in .NET Core 3.0.

With .NET Core, it has always been possible to publish self-contained apps that include everything needed to run your code, without requiring .NET to be installed on the deployment target. In some cases, the app only requires a small subset of the framework to function and could potentially be made much smaller by including only the used libraries.

We use the IL linker to scan the IL of your application to detect which code is actually required, and then trim unused framework libraries. This can significantly reduce the size of some apps. Typically, small tool-like console apps benefit the most as they tend to use fairly small subsets of the framework and are usually more amenable to trimming.

To use the linker:

  • Set the PublishTrimmed property to true.
  • Publish using an explicit RuntimeIdentifier.

Here’s an example:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>netcoreapp3.0</TargetFramework>
    <PublishTrimmed>true</PublishTrimmed>
  </PropertyGroup>
</Project>

And publish using the following command:

dotnet publish -r win-x64 -c Release

Note: The RuntimeIdentifier can be set to another operating system or chip. It can also be set in the project file.

The publish output will include a subset of the framework libraries, depending on what the application code calls. For a helloworld app, the linker reduces the size from ~68MB to ~28MB.

Applications or frameworks (including ASP.NET Core and WPF) that use reflection or related dynamic features will often break when trimmed, because the linker doesn’t know about this dynamic behavior and usually can’t determine which framework types will be required for reflection at run time. To trim such apps, you need to tell the linker about any types needed by reflection in your code, and in any packages or frameworks that you depend on. Be sure to test your apps after trimming. We are working on improving this experience for .NET 5.

For more information about the IL Linker, see the documentation, or visit the mono/linker repo.

Note: In previous versions of .NET Core, ILLink.Tasks was shipped as an external NuGet package and provided much of the same functionality. It is no longer supported – please update to the .NET Core 3.0 SDK and try the new experience!

The linker and ReadyToRun compiler can be used for the same application. In general, the linker makes your application smaller, and then the ready-to-run compiler will make it a bit larger again, but with a significant performance win. It is worth testing in various configurations to understand the impact of each option.

Publishing single-file executables

You can now publish a single-file executable with dotnet publish. This form of single EXE is effectively a self-extracting executable. It contains all dependencies, including native dependencies, as resources. At startup, it copies all dependencies to a temp directory, and loads them for there. It only needs to unpack dependencies once. After that, startup is fast, without any penalty.

You can enable this publishing option by adding the PublishSingleFile property to your project file or by adding a new switch on the commandline.

To produce a self-contained single EXE application, in this case for 64-bit Windows:

dotnet publish -r win10-x64 /p:PublishSingleFile=true

Note: The RuntimeIdentifier can be set to another operating system or chip. It can also be set in the project file.

See Single file bundler for more information.

Assembly trimmer, ahead-of-time compilation (via crossgen) and single file bundling are all new features in .NET Core 3.0 that can be used together or separately.

We expect that some of you will prefer single exe provided by an ahead-of-time compiler, as opposed to the self-extracting-executable approach that we are providing in .NET Core 3.0. The ahead-of-time compiler approach will be provided as part of the .NET 5 release.

dotnet build now copies dependencies

dotnet build now copies NuGet dependencies for your application from the NuGet cache to your build output folder during the build operation. Until this release,those dependencies were only copied as part of dotnet publish. This change allows you to xcopy your build output to different machines.

There are some operations, like linking and razor page publishing that require publishing.

.NET Core Tools — local installation

.NET Core tools has been updated to allow local installation. They have advantages over global tools, which were added in .NET Core 2.1.

Local installation enables the following:

  • Limit the scope by which a tool can be used.
  • Always use a specific version of the tool, which might differ from a globally-installed tool or another local installation. This is based on the version in the local tools manifest file.
  • Launched with dotnet, like in dotnet mytool.

Note: See Local Tools Early Preview Documentation for more information.

.NET Core SDK installers will now Upgrade in Place

The .NET Core SDK MSI installers for Windows will start upgrading patch versions in place. This will reduce the number of SDKs that are installed on both developer and production machines.

The upgrade policy will specifically target .NET Core SDK feature bands. Feature bands are defined in hundreds groups in the patch section of the version number. For example, 3.0.101 and 3.0.201 are versions in two different feature bands while 3.0.101 and 3.0.199 are in the same feature band.

This means when .NET Core SDK 3.0.101 becomes available and is installed, .NET Core SDK 3.0.100 will be removed from the machine if it exists. When .NET Core SDK 3.0.200 becomes available and is installed on the same machine, .NET Core SDK 3.0.101 will not be removed. In that situation, .NET Core SDK 3.0.200 will still be used by default, but .NET Core SDK 3.0.101 (or higher .1xx versions) will still be usable if it is configured for use via global.json.

This approach aligns with the behavior of global.json, which allows roll forward across patch versions, but not feature bands of the SDK. Thus, upgrading via the SDK installer will not result in errors due to a missing SDK. Feature bands also align with side by side Visual Studio installations for those users that install SDKs for Visual Studio use.

For more information, please check out:

.NET Core SDK Size Improvements

The .NET Core SDK is significantly smaller with .NET Core 3.0. The primary reason is that we changed the way we construct the SDK, by moving to purpose-built “packs” of various kinds (reference assemblies, frameworks, templates). In previous versions (including .NET Core 2.2), we constructed the SDK from NuGet packages, which included many artifacts that were not required and wasted a lot of space.

.NET Core 3.0 SDK Size (size change in brackets)

Operating System Installer Size (change) On-disk Size (change)
Windows 164MB (-440KB; 0%) 441MB (-968MB; -68.7%)
Linux 115MB (-55MB; -32%) 332MB (-1068MB; -76.2%)
macOS 118MB (-51MB; -30%) 337MB (-1063MB; -75.9%)

The size improvements for Linux and macOS are dramatic. The improvement for Windows is smaller because we have added WPF and Windows Forms as part of .NET Core 3.0. It’s amazing that we added WPF and Windows Forms in 3.0 and the installer is still (a little bit) smaller.

You can see the same benefit with .NET Core SDK Docker images (here, limited to x64 Debian and Alpine).

Distro 2.2 Size 3.0 Size
Debian 1.74GB 706MB
Alpine 1.48GB 422MB

You can see how we calculated these file sizes in .NET Core 3.0 SDK Size Improvements. Detailed instructions are provided so that you can run the same tests in your own environment.

Docker Publishing Update

Microsoft teams are now publishing container images to the Microsoft Container Registry (MCR). There are two primary reasons for this change:

  • Syndicate Microsoft-provided container images to multiple registries, like Docker Hub and Red Hat.
  • Use Microsoft Azure as a global CDN for delivering Microsoft-provided container images.

On the .NET team, we are now publishing all .NET Core images to MCR. As you can see from the links (if you click on it), we continue to have “home pages” on Docker Hub. We intend for that to continue indefinitely. MCR does not offer such pages, but relies of public registries, like Docker Hub, to provide users with image-related information.

The links to our old repos, such as microsoft/dotnet and microsoft/dotnet-nightly now forward to the new locations. The images that existed at those locations still exists and will not be deleted.

We will continue servicing the floating tags in the old repos for the supported life of the various .NET Core versions. For example, 2.1-sdk, 2.2-runtime, and latest are examples of floating tags that will be serviced. A three-part version tag like 2.1.2-sdk will not be serviced, which was already the case. We will only be supporting .NET Core 3.0 images in MCR.

For example, the correct tag string to pull the 3.0 SDK image now looks like the following:

mcr.microsoft.com/dotnet/core/sdk:3.0

The new MCR string will be used with both docker pull and in Dockerfile FROM statements.

See .NET Core Images now available via Microsoft Container Registry for more information.

SDK Docker Images Contain PowerShell Core

PowerShell Core has been added to the .NET Core SDK Docker container images, per requests from the community. PowerShell Core is a cross-platform (Windows, Linux, and macOS) automation and configuration tool/framework that works well with your existing tools and is optimized for dealing with structured data (e.g. JSON, CSV, XML, etc.), REST APIs, and object models. It includes a command-line shell, an associated scripting language and a framework for processing cmdlets.

You can try out PowerShell Core, as part of the .NET Core SDK container image, by running the following Docker command:

docker run --rm mcr.microsoft.com/dotnet/core/sdk:3.0 pwsh -c Write-Host "Hello Powershell"

There are two main scenarios that having PowerShell inside the .NET Core SDK container image enables, which were not otherwise possible:

Example syntax for launching PowerShell for a (volume-mounted) containerized build:

  • docker run -it -v c:\myrepo:/myrepo -w /myrepo mcr.microsoft.com/dotnet/core/sdk:3.0 pwsh build.ps1
  • docker run -it -v c:\myrepo:/myrepo -w /myrepo mcr.microsoft.com/dotnet/core/sdk:3.0 ./build.ps1

For the second example to work, on Linux, the .ps1 file needs to have the following pattern, and needs to be formatted with Unix (LF) not Windows (CRLF) line endings:

#!/usr/bin/env pwsh
Write-Host "test"

If you are new to PowerShell and would like to learn more, we recommend reviewing the getting started documentation.

Note: PowerShell Core is now available as part of .NET Core 3.0 SDK container images. It is not part of the .NET Core 3.0 SDK.

Red Hat Support

In April 2015, we announced that .NET Core would be coming to Red Hat Enterprise Linux. Through an excellent engineering partnership with Red Hat, .NET Core 1.0 appeared as a component available in the Red Hat Software Collections, June 2016. Working with Red Hat engineers, we have learned (and continue to learn!) much about the releasing software to the Linux community.

Over the last four years, Red Hat has shipped many .NET Core updates and significant releases, such as 2.1 and 2.2, on the same day as the Microsoft. With .NET Core 2.2, Red Hat expanded their .NET Core offerings to include OpenShift platforms. With the release of RHEL 8, we are excited to have .NET Core 2.1 and soon, 3.0, available in the Red Hat Application Streams.

Closing

.NET Core 3.0 is a major new release of .NET Core, and includes a vast set of improvements. We recommend that you start adopting .NET Core 3.0 as soon as you can. It greatly improves .NET Core in many ways, like the massive reduction in size of the SDK, and by greatly improving support for key scenarios like containers and Windows desktop applications. There are also many small improvements that were not included in this post, that you are sure to benefit from over time.

Please share your feedback with us, either in the coming days, weeks or months. We hope you enjoy it. We had a lot of fun making it for you.

If you still want to read more, the following recent posts are recommended reading:

The post Announcing .NET Core 3.0 appeared first on .NET Blog.

Announcing F# 4.7

$
0
0

We’re excited to announce general availability of F# 4.7 in conjunction with the .NET Core 3.0 release! In this post, I’ll show you how to get started, explain everything in F# 4.7 and give you a sneak peek at what we’re doing for the next version of F#.

F# 4.7 is another incremental release of F# with a focus on infrastructural changes to the compiler and core library and some relaxations on previously onerous syntax requirements.

F# 4.7 was developed entirely via an open RFC (requests for comments) process. The F# community has offered very detailed feedback in discussions for this version of the language. You can view all RFCs that correspond with this release here:

Get started

First, install either:

If you are a Visual Studio user, you will get an appropriate .NET Core installed by default. Once you have installed either .NET Core or Visual Studio 2019, you can use F# 4.7 with Visual Studio, Visual Studio for Mac, or Visual Studio Code with Ionide.

FSharp.Core now targets .NET Standard 2.0

Starting with FSharp.Core 4.7.0 and F# 4.7, we’re officially dropping support for .NET Standard 1.6. Now that FSharp.Core targets .NET Standard 2.0, you can enjoy a few new goodies on .NET Core:

  • Simpler dependencies, especially if using a tool like Paket
  • FromConverter and ToConverter static methods on FSharpFunc<'T, 'TResult>
  • Implicit conversions between FSharpFunc<'T, 'TResult> and Converter<'T, 'TResult>
  • The FuncConvert.ToFSharpFunc<'T> method
  • Access to the MatchFailureException type
  • The WebExtensions namespace for working with older web APIs in an F#-friendly way

Additionally, the FSharp.Core API surface area has expanded to better support parallel and sequential asynchronous computations:

  • Async.Parallel has an optional maxDegreesOfParallelism parameter so you can tune the degree of parallelism used
  • Async.Sequential to allow sequential processing of async computations

Thanks to Fraser Waters for contributing the new FSharp.Core additions.

Support for LangVersion

F# 4.7 introduces the ability to tune your effective language version with your compiler. We’re incredibly excited about this feature, because it allows us to deliver preview features alongside released features for any given compiler release.

If you’re interested in trying out preview features and giving feedback early, it’s very easy to get started. Just set the following property in your project file:

Once you save the project file, the compiler will now give you access to all preview features that shipped with that compiler.

When using F# in preview versions of .NET Core and/or Visual Studio, the language version will be set to preview by default.

The lowest-supported language version is F# 4.6. We do not plan on retrofitting language version support for F# 4.5 and lower.

Implicit yields

In the spirit of making things easier, F# 4.7 introduces implicit yields for lists, arrays, sequences, and any Computation Expression that defines the Yield, Combine, Delay, and Zero members.

A longstanding issue with learning F# has been the need to always specify the yield keyword in F# sequence expressions. Now you can delete all the yield keywords, since they’re implicit!

This makes F# sequence expressions align with list and array expressions.

But that’s not all! Prior to F# 4.7, even with lists and arrays, if you wanted to conditionally generate values it was a requirement to specify yield everywhere, even if you only had one place you did it. All the yield keywords can now be removed:

This feature was inspired by Fable programs that use F# list expressions as HTML templating DSLs.

Syntax relaxations

There are two major relaxations for F# syntax added in F# 4.7. Both should make F# code easier to write, especially for beginners.

No more required double underscore

Prior to F# 4.7, if you wanted to specify member declarations and you didn’t want to name the ‘this’ identifier on F# objects, you had to use a double underscore. Now, you can only specify a single underscore, which previous language versions would reject:

This same rule has been relaxed for C-style for loops where the indexer is not meaningful:

Thanks to Gustavo Leon for contributing this feature.

Indentation relaxations for parameters passed to constructors and static methods

Another annoyance with previous F# compilers was a requirement to indent parameters to constructors or static methods. This was due to an old rule in the compiler where the first parameter determined the level of indentation required for the rest of the parameters. This is now relaxed:

Preview features

As I mentioned previously, F# 4.7 introduces the concept of an effective language version for the compiler. In the spirit of shipping previews as early as possible, we’ve included two new preview features: nameof and opening of static classes.

Nameof

The nameof function has been of the most-requested feature to add to F#. It’s very convenient when you want to log the names of things (like parameters or classes) and have the name change as you’d expect if you refactor those symbols to use different names over time. We’re still not 100% resolute on the design of it, but the core functionality is good enough that we’d love people to try it out and give us feedback. Here’s a little taste of what you can do with it:

You can also contribute to its design by proposing changes to the corresponding RFC.

Open static classes

Much like nameof, opening of static classes has been requested a lot. Not only does it allow better usage of C# APIs that assume the ability to open static classes, it can also improve F# DSLs. However, we’re also not 100% resolute on its overall design. Here’s a little taste of what it’s like:


You can also contribute to its design by proposing changes to the corresponding RFC.

F# Interactive for .NET Core Preview

Starting with F# 4.7 and .NET Core 3, you can now use F# interactive (FSI) from .NET Core! Just open a command line and type dotnet fsi to get started.

The FSI experience for .NET Core is now a very, very stable preview. There are still some quirks with dependency resolution when pulling in packages and their transitive references. We’re addressing these by adding #r “nuget:package-name” support for FSI, and we’re hoping that you’ll transition away from manually referencing third-party .dlls and instead using packages as the unit of reference for FSI.

This package management support is still only available in nightly builds of the compiler. It will become available for general usage in forthcoming support for Jupyter Notebooks via the .NET Kernel and in the first preview of .NET 5.

Updates to F# tools for Visual Studio

The Visual Studio 2019 update 16.3 release corresponds with F# 4.7 and .NET Core 3. In this release, we’ve made tooltips a bit nicer and fixed some longstanding issues in the compiler and tools that affect your experience in Visual Studio. We also spent a lot of time doing more infrastructural work to make the F# integration with Roslyn significantly more stable than it was in the past.

Record definition tooltips use a more canonical formatting:

Anonymous Records also do the same:

And record value output in FSI also uses a more canonical form:

Properties with explicit get/set modifiers will also reflect those modifiers in tooltips:

Looking back at the past year or so of F# evolution

The past year (plus a few months) has seen a lot of additions to the F# language and tools. We’ve shipped:

  • F# 4.5, F# 4.6, and now F# 4.7 with 14 new language features between the three of them
  • 6 updates to the Visual Studio tools for F#
  • Massive performance improvements to F# tooling for larger codebases
  • 2 preview features for the next version of F#
  • A revamped versioning scheme for FSharp.Core
  • A new home for F# OSS development under the .NET Foundation

It’s been quite a rush, and while the sheer number of updates and fundamental shifts to F#, we’re planning on ramping up these efforts!

Looking head towards F# 5 and .NET 5

As .NET undergoes a monumental shift towards .NET 5, F# will also feature a bit of a shift. While F# is a general-purpose language – the functional programming language for .NET – it also has a strong heritage of being used for “analytical” workloads: processing data, doing numerical work, data science and machine learning, etc. We feel that F# is positioned extremely well to continue this path, and we intend on emphasizing features that can align with these workloads more.

Some of the concrete things we’ll focus on is making F# a first-class language for Jupyter Notebooks via the .NET Kernel. We’ll also emphasize language features that make it easier to work with collections of data.

I like to think of these things as being “in addition to” everything F# is focused on so far: first-class .NET support, excellent tooling, wonderful features that make general purpose F# programming great, and now an influx of work aligned with “analytical” programming. We’re incredibly excited about the work ahead of us, and we hope you’ll also contribute in the way you see best.

Cheers, and happy hacking!

The post Announcing F# 4.7 appeared first on .NET Blog.

Announcing Entity Framework Core 3.0 and Entity Framework 6.3 General Availability

$
0
0

We are extremely excited to announce the general availability of EF Core 3.0and EF 6.3 on nuget.org.

The final versions of .NET Core 3.0 and ASP.NET Core 3.0 are also available now.

How to get EF Core 3.0

EF Core 3.0 is distributed exclusively as a set of NuGet packages. For example, to add the SQL Server provider to your project, you can use the following command using the dotnet tool:

dotnet add package Microsoft.EntityFrameworkCore.SqlServer --version 3.0.0

When upgrading applications that target older versions of ASP.NET Core to 3.0, you also have to add the EF Core packages as an explicit dependency.

Also starting in 3.0, the dotnet ef command-line tool is no longer included in the .NET Core SDK. Before you can execute EF Core migration or scaffolding commands, you’ll have to install this package as either a global or local tool. To install the final version of our 3.0.0 tool as a global tool, use the following command:

dotnet tool install --global dotnet-ef --version 3.0.0

Specifying the version in the command is now optional. If you omit it, dotnet tool install will automatically install the latest stable version, which is right now 3.0.0.

It’s possible to use this new version of dotnet ef with projects that use older versions of the EF Core runtime. However, older versions of the tool will not work with EF Core 3.0.

What’s new in EF Core 3.0

Including major features, minor enhancements, and bug fixes, EF Core 3.0 contains more than 600 product improvements. Here are some of the most important ones:

LINQ overhaul

We rearchitected our LINQ provider to enable translating more query patterns into SQL, generating efficient queries in more cases, and preventing inefficient queries from going undetected. The new LINQ provider is the foundation over which we’ll be able to offer new query capabilities and performance improvements in future releases, without breaking existing applications and data providers.

Restricted client evaluation

The most important design change has to do with how we handle LINQ expressions that cannot be converted to parameters or translated to SQL.

In previous versions, EF Core identified what portions of a query could be translated to SQL, and executed the rest of the query on the client. This type of client-side execution is desirable in some situations, but in many other cases it can result in inefficient queries.

For example, if EF Core 2.2 couldn’t translate a predicate in a Where() call, it executed an SQL statement without a filter, transferred all the rows from the database, and then filtered them in-memory:

var specialCustomers = 
  context.Customers
    .Where(c => c.Name.StartsWith(n) && IsSpecialCustomer(c));

That may be acceptable if the database contains a small number of rows but can result in significant performance issues or even application failure if the database contains a large number or rows.

In EF Core 3.0, we’ve restricted client evaluation to only happen on the top-level projection (essentially, the last call to Select()). When EF Core 3.0 detects expressions that can’t be translated anywhere else in the query, it throws a runtime exception.

To evaluate a predicate condition on the client as in the previous example, developers now need to explicitly switch evaluation of the query to LINQ to Objects:

var specialCustomers =
  context.Customers
    .Where(c => c.Name.StartsWith(n)) 
    .AsEnumerable() // switches to LINQ to Objects
    .Where(c => IsSpecialCustomer(c));

See the breaking changes documentation for more details about how this can affect existing applications.

Single SQL statement per LINQ query

Another aspect of the design that changed significantly in 3.0 is that we now always generate a single SQL statement per LINQ query. In previous versions, we used to generate multiple SQL statements in certain cases, like to translate Include() calls on collection navigation properties and to translate queries that followed certain patterns with subqueries. Although this was in some cases convenient, and for Include() it even helped avoid sending redundant data over the wire, the implementation was complex, it resulted in some extremely inefficient behaviors (N+1 queries), and there was situations in which the data returned across multiple queries could be inconsistent.

Similarly to client evaluation, if EF Core 3.0 can’t translate a LINQ query into a single SQL statement, it throws a runtime exception. But we made EF Core capable of translating many of the common patterns that used to generate multiple queries to a single query with JOINs.

Cosmos DB support

The Cosmos DB provider for EF Core enables developers familiar with the EF programing model to easily target Azure Cosmos DB as an application database. The goal is to make some of the advantages of Cosmos DB, like global distribution, “always on” availability, elastic scalability, and low latency, even more accessible to .NET developers. The provider enables most EF Core features, like automatic change tracking, LINQ, and value conversions, against the SQL API in Cosmos DB.

See the Cosmos DB provider documentation for more details.

C# 8.0 support

EF Core 3.0 takes advantage of a couple of the new features in C# 8.0:

Asynchronous streams

Asynchronous query results are now exposed using the new standard IAsyncEnumerable<T> interface and can be consumed using await foreach.

var orders = 
  from o in context.Orders
  where o.Status == OrderStatus.Pending
  select o;

await foreach(var o in orders)
{
  Process(o);
}

See the asynchronous streams in the C# documentation for more details.

Nullable reference types

When this new feature is enabled in your code, EF Core examines the nullability of reference type properties and applies it to corresponding columns and relationships in the database: properties of non-nullable references types are treated as if they had the [Required] data annotation attribute.

For example, in the following class, properties marked as of type string? will be configured as optional, whereas string will be configured as required:

public class Customer
{
  public int Id { get; set; }
  public string FirstName { get; set; }
  public string LastName { get; set; }
  public string? MiddleName { get; set; }
}

See nullable reference types in the C# documentation for more details.

Interception of database operations

The new interception API in EF Core 3.0 allows providing custom logic to be invoked automatically whenever low-level database operations occur as part of the normal operation of EF Core. For example, when opening connections, committing transactions, or executing commands.

Similarly to the interception features that existed in EF 6, interceptors allow you to intercept operations before or after they happen. When you intercept them before they happen, you are allowed to by-pass execution and supply alternate results from the interception logic.

For example, to manipulate command text, you can create an IDbCommandInterceptor:

public class HintCommandInterceptor : DbCommandInterceptor
{
  public override InterceptionResult ReaderExecuting(
    DbCommand command, 
    CommandEventData eventData, 
    InterceptionResult result)
  {
    // Manipulate the command text, etc. here...
    command.CommandText += " OPTION (OPTIMIZE FOR UNKNOWN)";
    return result;
  }
}

And register it with your DbContext:

services.AddDbContext(b => b
  .UseSqlServer(connectionString)
  .AddInterceptors(new HintCommandInterceptor()));

Reverse engineering of database views

Query types, which represent data that can be read from the database but not updated, have been renamed to keyless entity types. As they are an excellent fit for mapping database views in most scenarios, EF Core now automatically creates keyless entity types when reverse engineering database views.

For example, using the dotnet ef command-line tool you can type:

dotnet ef dbcontext scaffold "Server=(localdb)\mssqllocaldb;Database=Blogging;Trusted_Connection=True;" Microsoft.EntityFrameworkCore.SqlServer

And the tool will now automatically scaffold types for views and tables without keys:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
  modelBuilder.Entity<Names>(entity =>
  {
    entity.HasNoKey();
    entity.ToView("Names");
  });

  modelBuilder.Entity<Things>(entity =>
  {
    entity.HasNoKey();
  });
}

Dependent entities sharing a table with principal are now optional

Starting with EF Core 3.0, if OrderDetails is owned by Order or explicitly mapped to the same table, it will be possible to add an Order without an OrderDetails and all of the OrderDetails properties, except the primary key will be mapped to nullable columns.

When querying, EF Core will set OrderDetails to null if any of its required properties doesn’t have a value, or if it has no required properties besides the primary key and all properties are null.

public class Order
{
    public int Id { get; set; }
    public int CustomerId { get; set; }
    public OrderDetails Details { get; set; }
}

[Owned]
public class OrderDetails
{
    public int Id { get; set; }
    public string ShippingAddress { get; set; }
}

What’s new in EF 6.3

We understand that many existing applications use previous versions of EF, and that porting them to EF Core only to take advantage of .NET Core can require a significant effort. For that reason, we decided to port the newest version of EF 6 to run on .NET Core 3.0. The developer community also contributed to this release with several bug fixes and enhancements.

Here are some of the most notable improvements:

  • Support for .NET Core 3.0
    • The EF 6.3 runtime package now targets .NET Standard 2.1 in addition to .NET Framework 4.0 and 4.5.
    • The migration commands have been rewritten to execute out of process and work with SDK-style projects.
  • Support for SQL Server hierarchyid
  • Improved compatibility with Roslyn and NuGet PackageReference
  • Added the ef6.exe utility for enabling, adding, scripting, and applying migrations from assemblies. This replaces migrate.exe

There are certain limitations when using EF 6.3 in .NET Core. For example:

  • Data providers need to be also ported to .NET Core. We only ported the SQL Server provider, which is included in the EF 6.3 package.
  • Spatial support won’t be enabled with SQL Server because the spatial types aren’t enabled to work with .NET Core.
  • There’s currently no support for using the EF designer directly on .NET Core or .NET Standard projects.

For more details on the EF 6.3 release, and a workaround to the latter limitation, see What’s new in EF 6.3 in the product’s documentation.

What’s next: EF Core 3.1

The EF team is now focused on the EF Core 3.1 release, which is planned for later this year, and on making sure that the documentation for EF Core 3.0 is complete.

EF Core 3.1 will be a long-term support (LTS) release, which means it will be supported for at least 3 years. Hence the focus is on stabilizing and fixing bugs rather than adding new features and risky changes. We recommend that you adopt .NET Core 3.0 today and then adopt 3.1 when it becomes available. There won’t be breaking changes between these two releases.

The full set of issues fixed in 3.1 can be seen in our issue tracker. Here are some worth mentioning:

  • Fixes and improvements for issues recently found in the Cosmos DB provider
  • Fixes and improvements for issues recently found in the new LINQ implementation
  • Lots of regressions tests added for issues verified as fixed in 3.0
  • Test stability improvements
  • Code cleanup

The first preview of EF Core 3.1 will be available very soon.

Thank you

If you either sent code contributions or feedback for any of our preview releases, thanks a lot! You helped make EF Core 3.0 and EF 6.3 significantly better!

We hope everyone will now enjoy the results.

The post Announcing Entity Framework Core 3.0 and Entity Framework 6.3 General Availability appeared first on .NET Blog.


Joining the .NET Foundation Maturity Model Pilot

$
0
0

Joining the .NET Foundation Maturity Model Pilot

The .NET Foundation is starting a new pilot program to increase quality and user confidence in open source projects, using a new project maturity model. We’ve been working with the Technical Review Action Group at the Foundation to help shape the program. We’re happy to see the pilot being launched and that the .NET Team is participating in the project. For us, this includes the underlying .NET platform, and also the packages we release.

We get to talk with larger organizations frequently, both from the private and public sectors, about open source. On one end of the spectrum, we see enthusiastic adopters of open source and on the other, an “open source isn’t safe for our business” approach. We also see organizations at all points between, and listen to their feedback about their practices using (or not using) open source and why. There are merits for each pattern we see. A big part of our contribution to the pilot was generalizing the underlying reasons for those approaches, and validating that the new maturity model will provide benefit to these organizations, and make adoption of open source safer and easier for them.

This new pilot program is similar to programs already in place at other foundations, like Cloud Native Computing Foundation (CNCF) and Apache Foundation. It is great to see the .NET Foundation expanding its role and taking on some of the same kind of charter as other communities use. The track record at these foundations speaks for itself, so it makes sense to emulate their approach.

The .NET Foundation is proposing three new programs:

.NET Foundation Project Maturity Model

These programs should be great additions to the .NET ecosystem and solve challenges that need to be addressed. We’re interesting in helping each of these programs. For the project forge, in particular, we have at least one lab project that we’d be happy to donate to the Foundation as significant starter code for a new project, run by new maintainers.

The .NET Foundation Technical Action Group has set an ambitious plan to improve the .NET ecosystem, with these three new programs, and the guidance and structure that go along with them. We will do our part in supporting these programs and the Technical Actions Group. We’re looking forward to seeing these programs develop in our larger ecosystem.

The post Joining the .NET Foundation Maturity Model Pilot appeared first on .NET Blog.

ML.NET and Model Builder at .NET Conf 2019 (Machine Learning for .NET)

$
0
0

We are excited today to announce updates to Model Builder and improvements in ML.NET. You can learn more in the “What’s new in ML.NET?.” session at .NET Conf.

ML.NET is an open-source and cross-platform machine learning framework (Windows, Linux, macOS) for .NET developers.

ML.NET offers Model Builder Model Builder (a simple UI tool) and CLI to make it super easy to build custom ML Models using AutoML.

Using ML.NET, developers can leverage their existing tools and skillsets to develop and infuse custom AI into their applications by creating custom machine learning models for common scenarios like Sentiment Analysis, Recommendation, Image Classification and more!.

Following are the key highlights:

Model Builder updates

This release of Model Builder adds support for a new scenario and address many customer reported issues.

Model Builder screenshot 1

Feature engineering: In previous versions of Model Builder, after selecting your dataset, either from a file or from SQL Server, you only had the option to choose the column to predict (the Label). Any other columns in the dataset were automatically used to make the prediction (Features). Any columns that you did not want to include, you had to manipulate your dataset outside of Model Builder and then upload the modified dataset.

Feature engineering in Model Builder

Model consumption made easy!: In previous versions of Model Builder, there were numerous steps that you had to take after Model Builder’s code and model generation in order to consume the trained model in your app, including adding a reference to the generated library project, setting the model Copy to Output property to “Copy If Newer,” and adding the Microsoft.ML NuGet package to your app.

This has all been simplified and automated, so now all you have to do is copy + paste the code from the Next Steps in Model Builder, and then you can run your app and start making predictions!

Address customer feedback: This release also address many customer reported issues around installation errors, usability feedback and stability improvements and more. Learn more here.

ML.NET updates

This is a short summary of the features and enhancements added to ML.NET over the last few months.

Deep learning with ML.NET

Documentation updates

We have been working hard to add more documentation across tutorials, how-to guides, and more for Model Builder, CLI, and ML.NET Framework. We have also simplified the table of contents for the ML.NET Docs so that you can easily discover the content.

Documentation updates

New learn series for ML.NET

To help users get started with the basics of Machine Learning and ML.NET, we have created a set of learning videos. Please watch the series here.

ML.NET video series

Broad range of samples to learn from

We have added many scenarios for a variety of use cases with Machine Learning. You can learn and customize these samples for your scenario. Please find more samples on the ML.NET Samples GitHub repo.

ML.NET Samples

Try ML.NET and Model Builder today!

We are excited to release these updates for you, and we look forward to seeing what you will build with ML.NET. If you have any questions or feedback, you can ask them here for ML.NET and Model Builder.

Thanks and happy coding with ML.NET!

The ML.NET Team.

The post ML.NET and Model Builder at .NET Conf 2019 (Machine Learning for .NET) appeared first on .NET Blog.

Announcing .NET Core 3 Preview 2

$
0
0

Today, we are announcing .NET Core 3 Preview 2. It includes new features in .NET Core 3.0 and C# 8, in addition to the large number of new features in Preview 1. ASP.NET Core 3.0 Preview 2  is also released today. C# 8 Preview 2 is part of .NET Core 3 SDK, and was also released last week with Visual Studio 2019 Preview 2.

Download and get started with .NET Core 3 Preview 2 right now on Windows, macOS and Linux.

In case you missed it, we made some big announcements with Preview 1, including adding support for Windows Forms and WPF with .NET Core, on Windows, and that both UI frameworks will be open source. We also announced that we would support Entity Framework 6 on .NET Core, which will come in a later preview. ASP.NET Core is also adding many features including Razor components.

Please provide feedback on the release in the comments below or at dotnet/core #2263. You can see complete details of the release in the .NET Core 3 Preview 2 release notes.

.NET Core 3 will be supported in Visual Studio 2019, Visual Studio for Mac and Visual Studio Code. Visual Studio 2019 Preview 2 was released last week and has support for C# 8. The Visual Studio Code C# Extension (in pre-release channel) was also just updated to support C# 8.

C# 8

C# 8 is a major release of the language, as Mads describes in Do more with patterns in C# 8.0, Take C# 8.0 for a spin and Building C# 8.0. In this post, I’ll cover a few favorites that are new in Preview 2.

Using Declarations

Are you tired of using statements that require indenting your code? No more! You can now write the following code, which attaches a using declaration to the scope of the current statement block and then disposes the object at the end of it.

Switch Expressions

Anyone who uses C# probably loves the idea of a switch statement, but not the syntax. C# 8 introduces switch expressions, which enable the following: terser syntax, returns a value since it is an expression, and fully integrated with pattern matching. The switch keyword is “infix”, meaning the keyword sits between the tested value (here, that’s o) and the list of cases, much like expression lambdas. The following examples use the lambda syntax for methods, which integrates well with switch expressions but isn’t required.

You can see the syntax for switch expressions in the following example:

There are two patterns at play in this example. o first matches with the Point type pattern and then with the property pattern inside the {curly braces}. The _ describes the discard pattern, which is the same as default for switch statements.

You can go one step further, and rely on tuple deconstruction and parameter position, as you can see in the following example:

In this example, you can see you do not need to define a variable or explicit type for each of the cases. Instead, the compiler can match the tuple being testing with the tuples defined for each of the cases.

All of these patterns enable you to write declarative code that captures your intent instead of procedural code that implements tests for it. The compiler becomes responsible for implementing that boring procedural code and is guaranteed to always do it correctly.

There will still be cases where switch statements will be a better choice than switch expressions and patterns can be used with both syntax styles.

Async streams

Async streams are another major improvement in C# 8. They have been changing with each preview and require that the compiler and the framework libraries match to work correctly. You need .NET Core 3.0 Preview 2 to use async streams if you want to develop with either Visual Studio 2019 Preview 2 or the latest preview of the C# extension for Visual Studio Code. If you are using .NET Core 3.0 Preview 2 at the commandline, then everything will work as expected.

IEEE Floating-point improvements

Floating point APIs are in the process of being updated to comply with IEEE 754-2008 revision. The goal of this floating point project is to expose all “required” operations and ensure that they are behaviorly compliant with the IEEE spec.

Parsing and formatting fixes:

  • Correctly parse and round inputs of any length.
  • Correctly parse and format negative zero.
  • Correctly parse Infinity and NaN by performing a case-insensitive check and allowing an optional preceding + where applicable.

New Math APIs:

  • BitIncrement/BitDecrement — corresponds to the nextUp and nextDown IEEE operations. They return the smallest floating-point number that compares greater or lesser than the input (respectively). For example, Math.BitIncrement(0.0) would return double.Epsilon.
  • MaxMagnitude/MinMagnitude — corresponds to the maxNumMag and minNumMag IEEE operations, they return the value that is greater or lesser in magnitude of the two inputs (respectively). For example, Math.MaxMagnitude(2.0, -3.0) would return -3.0.
  • ILogB — corresponds to the logB IEEE operation which returns an integral value, it returns the integral base-2 log of the input parameter. This is effectively the same as floor(log2(x)), but done with minimal rounding error.
  • ScaleB — corresponds to the scaleB IEEE operation which takes an integral value, it returns effectively x * pow(2, n), but is done with minimal rounding error.
  • Log2 — corresponds to the log2 IEEE operation, it returns the base-2 logarithm. It minimizes rounding error.
  • FusedMultiplyAdd — corresponds to the fma IEEE operation, it performs a fused multiply add. That is, it does (x * y) + z as a single operation, there-by minimizing the rounding error. An example would be FusedMultiplyAdd(1e308, 2.0, -1e308) which returns 1e308. The regular (1e308 * 2.0) - 1e308 returns double.PositiveInfinity.
  • CopySign — corresponds to the copySign IEEE operation, it returns the value of x, but with the sign of y.

.NET Platform Dependent Intrinsics

We’ve added APIs that allow access to certain perf-oriented CPU instructions, such as the SIMD or Bit Manipulation instruction sets. These instructions can help achieve big performance improvements in certain scenarios, such as processing data efficiently in parallel. In addition to exposing the APIs for your programs to use, we have begun using these instructions to accelerate the .NET libraries too.

The following CoreCLR PRs demonstrate a few of the intrinsics, either via implementation or use:

For more information, take a look at .NET Platform Dependent Intrinsics, which defines an approach for defining this hardware infrastructure, allowing Microsoft, chip vendors or any other company or individual to define hardware/chip APIs that should be exposed to .NET code.

Introducing a fast in-box JSON Writer & JSON Document

Following the introduction of the JSON reader in preview1, we’ve added System.Text.Json.Utf8JsonWriter and System.Text.Json.JsonDocument.

As described in our System.Text.Json roadmap, we plan to provide a POCO serializer and deserializer next.

Utf8JsonWriter

The Utf8JsonWriter provides a high-performance, non-cached, forward-only way to write UTF-8 encoded JSON text from common .NET types like String, Int32, and DateTime. Like the reader, the writer is a foundational, low-level type, that can be leveraged to build custom serializers. Writing a JSON payload using the new Utf8JsonWriter is 30-80% faster than using the writer from Json.NET and does not allocate.

Here is a sample usage of the Utf8JsonWriter that can be used as a starting point:

The Utf8JsonWriter accepts IBufferWriter<byte> as the output location to synchronously write the json data into and you, as the caller, need to provide a concrete implementation. The platform does not currently include an implementation of this interface, but we plan to provide one that is backed by a resizable byte array. That implementation would enable synchronous writes, which could then be copied to any stream (either synchronously or asynchronously). If you are writing JSON over the network and include the System.IO.Pipelines package, you can leverage the Pipe-based implementation of the interface called PipeWriter to skip the need to copy the JSON from an intermediary buffer to the actual output.

You can take inspiration from this sample implementation of IBufferWriter<T>. The following is a skeleton array-backed concrete implementation of the interface:

JsonDocument

In preview2, we’ve also added System.Text.Json.JsonDocument which was built on top of the Utf8JsonReader. The JsonDocument provides the ability to parse JSON data and build a read-only Document Object Model (DOM) that can be queried to support random access and enumeration. The JSON elements that compose the data can be accessed via the JsonElement type which is exposed by the JsonDocument as a property called RootElement. The JsonElement contains the JSON array and object enumerators along with APIs to convert JSON text to common .NET types. Parsing a typical JSON payload and accessing all its members using the JsonDocument is 2-3x faster than Json.NET with very little allocations for data that is reasonably sized (i.e. < 1 MB).

Here is a sample usage of the JsonDocument and JsonElement that can be used as a starting point:

GPIO Support for Raspberry Pi

We added initial support for GPIO with Preview 1. As part of Preview 2, we have released two NuGet packages that you can use for GPIO programming.

The GPIO Packages includes APIs for GPIO, SPI, I2C and PWM devices. The IoT bindings package includes device bindings for various chips and sensors, the same ones available at dotnet/iot – src/devices.

Updated serial port APIs were announced as part of Preview 1. They are not part of these packages but are available as part of the .NET Core platform.

Local dotnet tools

Local dotnet tools have been improved in Preview 2. Local tools are similar to dotnet global tools but are associated with a particular location on disk. This enables per-project and per-repository tooling. You can read more about them in the .NET Core 3.0 Preview 1 post.

In this preview, we have added 2 new commands:

  • dotnet new tool-manifest
  • dotnet tool list

To add local tools, you need to add a manifest that will define the tools and versions available. The dotnet new tool-manifest command automates creation of the manifest required by local tools. After this file is created, you can add tools to it via dotnet tool install <packageId>.

The command dotnet tool list lists local tools and their corresponding manifest. The command dotnet tool list -g lists global tools.

There was a change in .NET Core Local Tools between .NET Core 3.0 Preview 1 and .NET Core 3.0 Preview 2. If you tried out local tools in Preview 1 by running a command like dotnet tool restore or dotnet tool install, you need to delete your local tools cache folder before local tools will work correctly in Preview 2. This folder is located at:

On mac, Linux: rm -r $HOME/.dotnet/toolResolverCache

On Windows: rmdir /s %USERPROFILE%\.dotnet\toolResolverCache

If you do not delete this folder, you will receive an error.

Assembly Unloadability

Assembly unloadability is a new capability of AssemblyLoaderContext. This new feature is largely transparent from an API perspective, exposed with just a few new APIs. It enables a loader context to be unloaded, releasing all memory for instantiated types, static fields and for the assembly itself. An application should be able to load and unload assemblies via this mechanism forever without experiencing a memory leak.

We expect this new capability to be used for the following scenarios:

  • Plugin scenarios where dynamic plugin loading and unloading is required.
  • Dynamically compiling, running and then flushing code. Useful for web sites, scripting engines, etc.
  • Loading assemblies for introspection (like ReflectionOnlyLoad), although MetadataLoadContext (released in Preview 1) will be a better choice in many cases.

More information:

Assembly unloading requires significant care to ensure that all references to managed objects from outside a loader context are understood and managed. When the loader context is requested to be unloaded, any outside references need to have been unreferenced so that the loader context is self-consistent only to itself.

Assembly unloadability was provided in the .NET Framework by Application Domains (AppDomains), which are not supported with .NET Core. AppDomains had both benefits and limitations compared to this new model. We consider this new loader context-based model to be more flexible and higher performance when compared to AppDomains.

Windows Native Interop

Windows offers a rich native API, in the form of flat C APIs, COM and WinRT. We’ve had support for P/Invoke since .NET Core 1.0, and have been adding the ability to CoCreate COM APIs and Activate WinRT APIs as part of the .NET Core 3.0 release. We have had many requests for these capabilities, so we know that they will get a lot of use.

Late last year, we announced that we had managed to automate Excel from .NET Core. That was a fun moment. You can now try this same demo yourself with the Excel demo sample. Under the covers, this demo is using COM interop features like NOPIA, object equivalence and custom marshallers.

Managed C++ and WinRT interop are coming in a later preview. We prioritized getting COM interop built first.

WPF and Windows Forms

The WPF and Windows Forms teams opened up their repositories, dotnet/wpf and dotnet/winforms, respectively, on December 4th, the same day .NET Core 3.0 Preview 1 was released. Much of the last month, beyond holidays, has been spent interacting with the community, merging PRs, and responding to issues. In the background, they’ve been integrating WPF and Windows Forms into the .NET Core build system, including adopting the Arcade SDK. Arcade is an MSBuild SDK that exposes functionality which is needed to build the .NET Platform. The WPF team will be publishing more of the WPF source code over the coming months.

The same teams have been making a final set of changes in .NET Framework 4.8. These same changes have also been added to the .NET Core versions of WPF and Windows Forms.

Visual Studio support

Desktop development on .NET Core 3 requires Visual Studio 2019. We added the WPF and Windows Forms templates to the New Project Dialog to make it easier starting your new applications without using the command line.

The WPF and Windows Forms designer teams are continuing to work on an updated designer for .NET Core, which will be part of a Visual Studio 2019 Update.

MSIX Deployment for Desktop apps

MSIX is a new Windows app package format. It can be used to deploy .NET Core 3 desktop applications to Windows 10.

The Windows Application Packaging Project, available in Visual Studio 2019 preview 2, allows you to create MSIX packages with self-contained .NET Core applications.

Note: The .NET Core project file must specify the supported runtimes in the <RuntimeIdentifiers> property:

<RuntimeIdentifiers>win-x86;win-x64</RuntimeIdentifiers>

Install .NET Core 3.0 Previews on Linux with Snap

Snap is the preferred way to install and try .NET Core previews on Linux distributions that support Snap. At present, only X64 builds are supported with Snap. We’ll look at supporting ARM builds, too.

After configuring Snap on your system, run the following command to install the .NET Core SDK 3.0 Preview SDK.

sudo snap install dotnet-sdk --beta --classic

When .NET Core in installed using the Snap package, the default .NET Core command is dotnet-sdk.dotnet, as opposed to just dotnet. The benefit of the namespaced command is that it will not conflict with a globally installed .NET Core version you may have. This command can be aliased to dotnet with:

sudo snap alias dotnet-sdk.dotnet dotnet

Some distros require an additional step to enable access to the SSL certificate. See our Linux Setup for details.

Platform Support

.NET Core 3 will be supported on the following operating systems:

  • Windows Client: 7, 8.1, 10 (1607+)
  • Windows Server: 2012 R2 SP1+
  • macOS: 10.12+
  • RHEL: 6+
  • Fedora: 26+
  • Ubuntu: 16.04+
  • Debian: 9+
  • SLES: 12+
  • openSUSE: 42.3+
  • Alpine: 3.8+

Chip support follows:

  • x64 on Windows, macOS, and Linux
  • x86 on Windows
  • ARM32 on Windows and Linux
  • ARM64 on Linux

For Linux, ARM32 is supported on Debian 9+ and Ubuntu 16.04+. For ARM64, it is the same as ARM32 with the addition of Alpine 3.8. These are the same versions of those distros as is supported for X64. We made a conscious decision to make supported platforms as similar as possible between X64, ARM32 and ARM64.

Docker images for .NET Core 3.0 are available at microsoft/dotnet on Docker Hub. We are in the process of adopting Microsoft Container Registry (MCR). We expect that the final .NET Core 3.0 images will only be published to MCR.

Closing

Take a look at the .NET Core 3.0 Preview 1 post if you missed that. It includes a broader description of the overall release including the initial set of features, which are also included and improved on in Preview 2.

Thanks to everyone that installed .NET Core 3.0 Preview 1. We appreciate you trying out the new version and for your feedback. Please share any feedback you have about Preview 2.

Submit to the Applied F# Challenge!

$
0
0

This post was written by Lena Hall, a Senior Cloud Developer Advocate at Microsoft.

F# Software Foundation has recently announced their new initiative — Applied F# Challenge! We encourage you to participate and send your submissions about F# on Azure through the participation form.

Applied F# Challenge is a new initiative to encourage in-depth educational submissions to reveal more of the interesting, unique, and advanced applications of F#.

The motivation for the challenge is uncovering more of advanced and innovative scenarios and applications of F# we hear about less often:

We primarily hear about people using F# for web development, analytical programming, and scripting. While those are perfect use cases for F#, there are many more brilliant and less covered scenarios where F# has demonstrated its strength. For example, F# is used in quantum computing, cancer research, bioinformatics, IoT, and other domains that are not typically mentioned as often.

You have some time to think about the topic for your submission because the challenge is open from February 1 to May 20 this year.

What should you submit?

Publish a new article or an example code project that covers a use case of a scenario where you feel the use of F# to be essential or unique. The full eligibility criteria and frequently asked questions are listed in the official announcement.

There are multiple challenge categories you can choose to write about:

F# for machine learning and data science.
F# for distributed systems.
F# in the cloud: web, serverless, containers, etc.
F# for desktop and mobile development.
F# in your organization or domain: healthcare, finance, games, retail, etc.
F# and open-source development.
F# for IoT or hardware programming.
F# in research: quantum, bioinformatics, security, etc.
Out of the box F# topics, scenarios, applications, or examples.

Why should you participate in the challenge?

All submissions will receive F# stickers as a participation reward for contributing to the efforts of improving the F# ecosystem and raising awareness of F# strengths in advanced or uncovered use cases.

Participants with winning submissions in each category will also receive the title of a Recognized F# Expert by F# Software Foundation and a special non-monetary prize.

Each challenge category will be judged by the committees that include many notable F# experts and community leaders, including Don Syme, Rachel Blasucci, Evelina Gabasova, Henrik Feldt, Tomas Perticek, and many more.

As the participation form suggests, you will also have an opportunity to be included in a recommended speaker list by F# Software Foundation.

Spread the word

Help us spread the word about the Applied F# Challenge by encouraging others to participate with #AppliedFSharpChallenge hashtag on Twitter!

Announcing ML.NET 0.10 – Machine Learning for .NET

$
0
0

alt text

ML.NET is an open-source and cross-platform machine learning framework (Windows, Linux, macOS) for .NET developers. Using ML.NET, developers can leverage their existing tools and skillsets to develop and infuse custom AI into their applications by creating custom machine learning models.

ML.NET allows you to create and use machine learning models targeting common tasks such as classification, regression, clustering, ranking, recommendations and anomaly detection. It also supports the broader open source ecosystem by proving integration with popular deep-learning frameworks like TensorFlow and interoperability through ONNX. Some common use cases of ML.NET are scenarios like Sentiment Analysis, Recommendations, Image Classification, Sales Forecast, etc. Please see our samples for more scenarios.

Today we’re announcing the release of ML.NET 0.10. ( ML.NET 0.1 was released at //Build 2018). Note that ML.NET follows a semantic versioning pattern, so this preview version is 0.10. There will be additional versions such as 0.11 and 0.12 before we release v1.0.

This release focuses on the overall stability of the framework, continuing to refine the API, increase test coverage and as an strategic milestone, we have moved the IDataView components into a new and separated assembly under Microsoft.Data.DataView namespace so it will favor interoperability in the future.

The main highlights for this blog post are described below in further details:

IDataView as a shared type across libraries in the .NET ecosystem

The IDataView component provides a very efficient, compositional processing of tabular data (columns and rows) especialy made for machine learning and advanced analytics applications. It is designed to efficiently handle high dimensional data and large data sets. It is also suitable for single node processing of data partitions belonging to larger distributed data sets.

For further info on IDataview read the IDataView design principles

What’s new in v0.10 for IDataView

In ML.NET 0.10 we have segregated the IDataView component into a single assembly and NuGet package. This is a very important step towards the interoperability with other APIs and frameworks.

Why segregate IDataView from the rest of the ML.NET framework?

This is a very important milestone that will help the ecosystem’s interoperability between multiple frameworks and libraries from Microsoft of third parties. By seggregating IDataView, different libraries will be able to reference it and use it from their API and allow users to pass large volumes of data between two independent libraries.

For example, from ML.NET you can of course consume and produce IDataView instances. But what if you need to integrate with a different framework by creating an IDataView from another API such as any “Data Preparation framework” library? If those frameworks can simply reference a single NuGet package with just the IDataView, then you can directly pass data into ML.NET from those frameworks without having to copy the data into a format that ML.NET consumes. Also, the additional framework wouldn’t depend on the whole ML.NET framework but just reference a very clean package limited to the IDataView.

The image below is an aspirational approach when using IDataView across frameworks in the ecosystem:

alt text

Another good example would be any plotting/charting library in the .NET ecosystem that could consume data using IDataView. You could take data that was produced by ML.NET and feed it directly into the plotting library without that library having a direct reference to the whole ML.NET framework. There would be no need to copy, or change the shape of the data at all. And there is no need for this plotting library to know anything about ML.NET.

Basically, IDataView can be an exchange data format which allows producers and consumers to pass large amounts of data in a standarized way.

For additional info check the PR #2220

Support for multiple ‘feature columns’ in recommendations (FFM based)

In previous ML.NET releases, when using the Field-aware Factorization Machine (FFM) trainer (training algorithm) you could only provide a single feature column like in this sample app

In 0.10 release we’ve added support for multiple ‘feature columns’ in your training dataset when using an FFM trainer by allowing to specify those additional column names in the trainer’s ‘Options’ parameter as shown in the following code snippet:

var ffmArgs = new FieldAwareFactorizationMachineTrainer.Options();

// Create the multiple field names.
ffmArgs.FeatureColumn = nameof(MyObservationClass.MyField1); // First field.
ffmArgs.ExtraFeatureColumns = new[]{ nameof(MyObservationClass.MyField2), nameof(MyObservationClass.MyField3) }; // Additional fields.

var pipeline = mlContext.BinaryClassification.Trainers.FieldAwareFactorizationMachine(ffmArgs);

var model = pipeline.Fit(dataView);

You can see additional code example details in this code

Additional updates in v0.10 timeframe

Support for returning multiple predicted labels

Until ML.NET v0.9, when predicting (for instance with a multi-class classification model), you could only predict and return a single label. That’s an issue for many business scenarios. For instance, in an eCommerce scenario, you could want to automatically classify a product and assign it to multiple product categories instead of just a single category.

However, when predicting, ML.NET internally already had a list of the multiple possible predictions with a score/proability per each in the schema’s data, but the API was simply not returning the list of possible predicted labels but a single one.

Therefore, this improvement allows you to access the schema’s data so you can get a list of the predicted labels which can then be related to their scores/proabilities provided by the float[] Score array in your Prediction class, such as in this sample prediction class.

For additional info check this code example

Minor updates in 0.10

  • Introducing Microsoft.ML.Recommender NuGet name instead of Microsoft.ML.MatrixFactorization name: Microsoft.ML.Recommender] is a better naming for NuGet packages based on the scenario (Recommendations) instead of the trainer’s name (Microsoft.ML.MatrixFactorization).

  • Added support in TensorFlow for using using text and sparse input in TensorFlow: Specifically, this adds support for loading a map from a file through dataview by using ValueMapperTransformer. This provides support for additional scenarios like a Text/NLP scenario) in TensorFlowTransform where model’s expected input is vector of integers.

  • Added Tensorflow unfrozen models support in GetModelSchema: For a code example loading an unfrozen TensorFlow check it out here.

Breaking changes in ML.NET 0.10

For your convenience, if you are moving your code from ML.NET v0.9 to v0.10, you can check out the breaking changes list that impacted our samples.

Instrumented code coverage tools as part of the ML.NET CI systems

We have also instrumented code coverage tools (using https://codecov.io/) as part of our CI systems and will continue to push for stability and quality in the code.

You can check it out here which is also a link in the home page of the ML.NET repo:

alt text

Once you click on that link, you’ll see the current code coverage for ML.NET:

alt text

Explore the community samples and share yours!!

As part of the ML.NET Samples repo we also have a special Community Samples page pointing to multiple samples provided by the community. These samples are not maintained by Microsoft but are very interesting and cover additional scenarios not covered by us.

Here’s an screenshot of the current community samples:

alt text

There are pretty cool samples like the following:

‘Photo-Search’ WPF app running a TensorFlow model exported to ONNX format

alt text

UWP app using ML.NET

alt text

Other very interesting samples are:

Share your sample with the ML.NET community!

alt text

We encourage you to share your ML.NET demos and samples with the community by simply submitting its brief description and URL pointing to your GitHub repo or blog posts, into this repo issue “Request for your samples!”.

We’ll do the rest and publish it at the ML.NET Community Samples page!

Planning to go to production?

alt text

If you are using ML.NET in your app and looking to go into production, you can talk to an engineer on the ML.NET team to:

  1. Get help implementing ML.NET successfully in your application.
  2. Demo your app and potentially have it featured on the .NET Blog, dot.net site, or other Microsoft channel.

Fill out this form and someone from the ML.NET team will contact you.

Get started!

alt text

If you haven’t already get started with ML.NET here.

Next, going further explore some other resources:

We will appreciate your feedback by filing issues with any suggestions or enhancements in the ML.NET GitHub repo to help us shape ML.NET and make .NET a great platform of choice for Machine Learning.

Thanks and happy coding with ML.NET!

The ML.NET Team.

This blog was authored by Cesar de la Torre and Eric Erhardt plus additional contributions of the ML.NET team

Viewing all 4000 articles
Browse latest View live


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>