How difficult is CLRS?

While I started this reading project 6 months ago, one question has always been in my mind - how difficult is to read CLRS?

My background is not straightly from computer science.  The first few comments I heard is that "it is impossible to read CLRS as a book." .  

Yet during my reading, I have stumbled upon several individuals who tried to the similar things as I do.  That is to deep read the whole book once -> i.e. browse, read, finish all the exercises and (optional) research.  

None of them really finish such a feat (nor I would claim I come closed) but many of them can finish couple of chapters with what I consider as correct answers. 

If they can do it, how come most cannot?    I am not inclined to blame talent as the reason  because I always think if you are not talented, you can simply learn better.  

When I start a course in famous extension school, CLRS is listed in two advanced Data Structure courses.  They are in two different topics.   Then I start to realize what really happens.  

      CLRS is probably not difficult but it will take almost 6-7 graduate courses to cover the whole book. 

And of course, most do not have the time to take 1-2 graduate courses because they can be very time-consuming.  And time is precisely what many people perceive they have little. 

But bear that in mind, CLRS is completely readable if you are used to read Mathematical text and long books. (If you are not, are you really a programmer?), the rest is just patience and persistence.