That low level players you're speaking off could be playing under their second account. =p
Here's the thing. I really like the ranking system in Halo 2 & 3, and it had kept me playing much much longer than Reach. But it will never be a deal breaker for me in buying a Halo game.
After playing COD games for so long, I do miss those moments where I'd say to myself, "**** there's a level 49 here." You know damn well that if you're seeing a group of 47-49s in Halo 3 ranked matches, you better bring your best game and prepare for the worst.
Yes there were some terrible high ranked players (even myself at level 48, 2nd account, had my own bad days), but from experience, matches were mostly competitive for me.
I do hate being grouped with people in Halo 3 that take ranking so seriously. When losing, they're some of the most angry people I've seen on XBL.
So yeah from reading around the forums, I'm glad there will be some sort of match making based on skill.