Probably you have a broken Music file. I had the same issue with my library. You can click on the working indicator in the upper right corner to see the currently scanned file. After I deleted the broken file my 40 GB library was scanned without Problems.
I followed your advice and my library is now scanned! Turns out it was a Christmas song that it kept crashing on. I didn't even realise I still had Christmas songs on there... Thanks for the suggestion!