To What Extent Does Google Use Visitor Time vs. Links in Page Rank / SERPs?

I think I remember reading Matt Cutts or someone else from google stating maybe 6 months back that Page Rank and/or SERP results don't take into account the attention users give to pages directly (by tracking user activity), but only indirectly (by making links to appreciated pages).  The argument was that it was too easy to create pseudo-software-solutions that could fake user visits.  I just tried to find a reference to that but I failed.

However, I believe that something like this does happen.  So far, I haven't found good evidence of it - maybe later - but I did just notice Mr. Eric Schmidt's comments in a recent talk where he states quite clearly that google has good data on user surfing (and he didn't backstep and say, "but it's probably unreliable data" so I think he believes there is good value in the data, and it may be used already... or soon ... what do you think?  See for yourself Eric Schmidt's comments:  (In case the timestamp doesn't embed properly in the video, here's the direct link to the relevant question: http://www.youtube.com/watch?v=orAJ-YD9FhA#t=52m30s)