One problem I find though is that every time I create a new project or solution I end up installing the same old NuGet packages. By default NuGet downloads the source code for these packages into a new "packages" folder in your project/solution root directory. This means I have the same source code downloaded multiple times, in multiple locations.
This can easily be fixed as you can configure NuGet to change where it stores packages for your projects and solutions. Simply create a file called "nuget.config" and drop it in your project/solution root directory. In order to set where NuGet stores your project packages add the following settings element to the file.
(C:\NuGetPackages is a bad example; you would probably want to use a relative path)
I find it really useful to have all of my projects and solutions share a single NuGet package store as it saves having duplicate copies of packages littered around. It also helps if you don’t intend to keep the packages in source control as you can move your NuGet package store out of your source control folder structure. Don’t worry about different projects needing different versions of the same package. NuGet can maintain multiple versions of the same package so you can still upgrade certain projects and not others.
N.B. I find using a centralized NuGet package store works really well with projects that also enable NuGet Package Restore (http://docs.nuget.org/docs/workflows/using-nuget-without-committing-packages). It is quite easy to tweak to reflect the change in package store directory.