Strong dialects, overlapping speakers, and very low signal-to-noise audio ratios are first order determinants of difficult audio. Several other factors such as audio recording quality, background noise and unscripted speech are also key contributors. If a file has only one of these factors and it persists for a small part of the total file duration, it is not likely to impact the difficulty rating. However, as the number of negative factors - and their time duration - increases, this will cause the file to become increasingly difficult. And, if these factors vary considerably throughout a file, that can have a significant additional impact on difficulty.
Check out our Audio Recording Guidelines for best practices on recording high quality audio.