I suddenly feel I am quite elated with the simple questions I solved in the Data Structure class for the famous extension school. Famous extension school is famous because its programs are one of the top of the world. But famous extension school is still a worthwhile place to study for a moment because it charges a large sum of fee for courses. Studying there may not be the most-valued thing in the world but the education is decent. You can think of its graduate program as at least in the top of the 2nd tier.
Some of its graduate exhibit a trait which I dislike : that is to claim they are graduated from the university itself. But extension school is still an extension school. Its quality is still poorer than the core. It's pretentious to claim you graduate from the core which only a few very talented can do. So I always refer it to the "extension school" because I don't like to brat with the name.
In such a context, being able to finish the homework with ease is a pretty encouraging thing. That might mean I might really able to apply jobs in some Big shots company. But well, let's wait and see. Programming skill is something it takes time to refine.
Another interesting feeling I have when I walked home is that programming could be something fun and interesting. Unlike work in my job which is dull and super-uncreative, a lot of programming tasks are artistic and challenging. This makes me feel happy about my choice to admit to the famous extension school. It really adds something positive to my life and balances out my rather s*ty work life.
Showing posts with label Thoughts in programming. Show all posts
Showing posts with label Thoughts in programming. Show all posts
Thursday, September 1, 2011
Sunday, June 5, 2011
Why people think "reading" CLRS completely is impossible?
If you search for "CLRS" in Google, in Jun 2011, you would find a link which ask "How many programmers recommending “Introduction to algorithms” by CLRS actually read whole book?"
This is a valid question. In a way, it is similar to "How many people recommending "Feynmann Lecture of Physics" actually read the whole book?" or "How many people actually read Dickens? Austen? Tolstoy? Dostoevsky?" or even "Tolkien?". In my way of thinking, I think people who have doubt about reading CLRS is possible don't really have issues in programming and algorithmic study. What they don't grasp is how reading and understanding works.
Most of us can read. The difference is mostly an issue about speed and understanding. Speed is easy to understand, sparing technicalities, the number of characters divided by the time you spent to read the whole book.
But what is understanding then? This is the core of the issues. This is a tough problem but in many types of work, there are simple answer. Let's work try to work out this kind of simple answer first. For example, in fictional work, one can ask simply what is the plot of the story. A deeper level, one can a detail chronology of all characters that include identifying their names, what they have done. We also want to identify all their interactions.
What is even deeper level? That could be analysis on the text of the work (Many did that on famous book such as ... well Bible). It could also be an intelligent guess of what the author *really* want to say. That's what critics do. It could be a comparison study of what this book compare with other similar books. Does it has philosophical meaning? Then philosophers would want to analyze its philosophical implications. Is it about mind and body? Monism vs Dualism? Theist vs Atheist? Is the book an expression of artistry? Then a thinking of whether this book is "beautiful". This is domain of beauty?
So what do we see here? Understanding of a book seems to be a representation of a book in different type of knowledge. Some representation is cruder (like just knowing the plot and the title). Some representation is certainly more complicated and difficult to understand.
Now a lot of contention of whether one can completely read a book lie on that it is not easy to agree what "reading" really means. This is not just an issue of computer science books. It is also an issue of any books. So we should clarify that first for our subject, CLRS. How do we define "understanding" of an algorithm?
In fact from the example of fiction, one can see a proper definition of understanding depends on what representation of understanding should we use. Many programmers, when they argue about this problem simply ignore this simple fact. That's fuzzy thinking. It's a big no-no for programmers. What are some other types of fuzzy thinking one can think of? I can imagine couple of others. Here is a big one.
Wrong. For a book like CLRS, even if you finish reading the book and complete all exercises yourself (meaning: without any help, you can answer all the exercises). You still need to learn. There are parts of CLRS need to be complemented by other books. Its sorting section is known to be less practical than Sedgewick AIC, its theoretical depth is known to be shallower than TAOCP.
I would say if you studied CLRS by heart, you would know around 100-200 standard algorithms. But you don't exactly know all of their variations. It's also impossible for *you* to change them because you don't necessarily understand why they survive these days. (Because their underlying principle is sound). By the definition, you need at least another read.
Back to the problem of understanding CLRS. So what can we say about what is understanding of an algorithm. One thing we established so far is that understanding is actually not just a simple "BEEEB" in our brain, then we understand. Understanding is a complicated object. We also see from the fiction example that understanding can mean different things to different people. So I will tell you my way of understanding the book. In the past, I have outlined a post titled Current Reading Methodologies for CLRS. I wrote that post with this thinking in mind.
So, in conclusion, I still whole-heartedly recommend you to read CLRS through. I, for one, has a cursory read until Chapter 15 in the last 2 months (what I called "fuzzy" read). I believe that means at around September, I will finish the so called "fuzzy" read in this September. My exercise read should be finished around 1.5 to 2 years later. But I still plan to finish it because just finishing the first two chapters give me a feeling that I have tremendously improved in programming. I think you can also be benefited in the process.
This is a valid question. In a way, it is similar to "How many people recommending "Feynmann Lecture of Physics" actually read the whole book?" or "How many people actually read Dickens? Austen? Tolstoy? Dostoevsky?" or even "Tolkien?". In my way of thinking, I think people who have doubt about reading CLRS is possible don't really have issues in programming and algorithmic study. What they don't grasp is how reading and understanding works.
Most of us can read. The difference is mostly an issue about speed and understanding. Speed is easy to understand, sparing technicalities, the number of characters divided by the time you spent to read the whole book.
But what is understanding then? This is the core of the issues. This is a tough problem but in many types of work, there are simple answer. Let's work try to work out this kind of simple answer first. For example, in fictional work, one can ask simply what is the plot of the story. A deeper level, one can a detail chronology of all characters that include identifying their names, what they have done. We also want to identify all their interactions.
What is even deeper level? That could be analysis on the text of the work (Many did that on famous book such as ... well Bible). It could also be an intelligent guess of what the author *really* want to say. That's what critics do. It could be a comparison study of what this book compare with other similar books. Does it has philosophical meaning? Then philosophers would want to analyze its philosophical implications. Is it about mind and body? Monism vs Dualism? Theist vs Atheist? Is the book an expression of artistry? Then a thinking of whether this book is "beautiful". This is domain of beauty?
So what do we see here? Understanding of a book seems to be a representation of a book in different type of knowledge. Some representation is cruder (like just knowing the plot and the title). Some representation is certainly more complicated and difficult to understand.
Now a lot of contention of whether one can completely read a book lie on that it is not easy to agree what "reading" really means. This is not just an issue of computer science books. It is also an issue of any books. So we should clarify that first for our subject, CLRS. How do we define "understanding" of an algorithm?
In fact from the example of fiction, one can see a proper definition of understanding depends on what representation of understanding should we use. Many programmers, when they argue about this problem simply ignore this simple fact. That's fuzzy thinking. It's a big no-no for programmers. What are some other types of fuzzy thinking one can think of? I can imagine couple of others. Here is a big one.
- "After I read the book for the first time, I will be able to use any algorithms in the book at ease."
Wrong. For a book like CLRS, even if you finish reading the book and complete all exercises yourself (meaning: without any help, you can answer all the exercises). You still need to learn. There are parts of CLRS need to be complemented by other books. Its sorting section is known to be less practical than Sedgewick AIC, its theoretical depth is known to be shallower than TAOCP.
I would say if you studied CLRS by heart, you would know around 100-200 standard algorithms. But you don't exactly know all of their variations. It's also impossible for *you* to change them because you don't necessarily understand why they survive these days. (Because their underlying principle is sound). By the definition, you need at least another read.
Back to the problem of understanding CLRS. So what can we say about what is understanding of an algorithm. One thing we established so far is that understanding is actually not just a simple "BEEEB" in our brain, then we understand. Understanding is a complicated object. We also see from the fiction example that understanding can mean different things to different people. So I will tell you my way of understanding the book. In the past, I have outlined a post titled Current Reading Methodologies for CLRS. I wrote that post with this thinking in mind.
- Level 1: knowing the algorithms and the basic terminology to describe the algorithm. e.g. you need to know the pseudocode of the insertion sort. In big O, it is a O(n^2) algorithm.
- Level 2: you can finish the exercises related the books. Why? If there were no exercises, all of us should simply jump to Level 3. That is to establish connections in between all related terms you learn. But exercise is an accelerated method. Especially in a well written book such as CLRS. Once you finish one exercise, you should have a feeling that you either know how to apply some material in the text. Or you are able to connection between some similar unrelated concepts. Or you are able to learn a new property of a certain object.
- Level 3: Establish connection on all the things we human know. For example in the case of big Oh. Can you differentiate it with soft Oh (The one which doesn't differentiate exponential difference). When you learn insertion sort, do you know its different variations? This starts to be a deep level because there are many possibility to connect. My "complementary exercises" are essentially about that. At this point, I am still trying to understand different variations of insertion sort. Sounds silly? Of course, if you ask me other slightly advanced things such as DP and graph algorithm, I also have working knowledge. It's just that they are not as deep. This is also a point you want to read other books as well. So even though I am reading CLRS, I read Sedgewick , CMath, TAOCP and Programming Pearls.
- Level 4: This starts to be the domain of research. What are the issues of in this area? This includes couple of things.
- What are the interesting new problems in this domain?
- What are the viewpoints in this domain?
- What are the unsolved problems in this domain?
- Is there any thing in this topic is "useful"? e.g. so efficient one should let every one should know?
- This is the moment you should feel confused.
- Level 5: Try to think of some solutions for the questions you have in Level 4. You might or might not able to think of any. If you do, don't waste it. Publish it. Tell other people.
So, in conclusion, I still whole-heartedly recommend you to read CLRS through. I, for one, has a cursory read until Chapter 15 in the last 2 months (what I called "fuzzy" read). I believe that means at around September, I will finish the so called "fuzzy" read in this September. My exercise read should be finished around 1.5 to 2 years later. But I still plan to finish it because just finishing the first two chapters give me a feeling that I have tremendously improved in programming. I think you can also be benefited in the process.
Saturday, June 4, 2011
Our lifes as programmers.
Sometimes you feel that you are going no where. Sometimes you feel you hit blocks which cannot be resolved. It is not because you haven't tried. No, I did, you try to be maximally capable, friendly and cooperative. The response is indifferent, cold and condescending. This is a disappointment. Because sometimes, no matter you tried, you will still have moment of bad lucks. Or simply bad beats as poker players called it. Those are the moments you have done the best you can do, but bad things just come to you.
The first thing you should do is to understand that this is not your fault. A pair of AA can get beat by Ax in Texas Holdem which every players know, it is a 93 to 7 shot. Sometimes you play well but you don't get pay well. That's just the way life is. Your boss, your peers as imperfect human beings, sometimes just make mistake. It is you to decide how to deal with this imperfect future.
The second thing is to make sure you still play well in the game. Don't go to drink because you are not happy for the day. Don't go to take drugs, smoke because you feel those are relieving things to do. They are counter-productive. Don't get emotional about these issues. If your boss or your colleague are obnoxious, eschew them. Make sure you make sure everything is correct in your domain. Control the outcome you can control.
The third thing to do is to keep on learning. There is always a better job waiting for you. Our discipline favor smart and hard-working people. There is always a better life outside the job you are working on. Every time you can get such a chance, you should ask for a salary adjustment. Why? Because we are hot. If you don't have a reason to stay in a place, leaving is not a bad idea. The best people in this country (also outside U.S.) move around. In that way, you can get benefit out of your jobs.
But remember, moving around different companies are not easy in our industry. You need to be good. So study, learn. Here the things I recommend you to learn:
1, Languages, Communications, Writing,
2, Math and basic computer science subjects. Top 10: Algorithms, Operating Systems, Compiler Design, Database, Discrete Mathematics, Probability and Random Process, Linear Algebra, Complexity Theory, Calculus, Machine Learning.
3, Management and Interpersonal Skill.
4, Musical Instruments,
5, Art,
6, Martial Arts.
1 and 2 are essential. All of them, when learn seriously, can open your mind. Being able to impress other people means a lot. 3 are accumulative. You just need to work and reflect some time. 4-6 will open your eyes.
The final thing to do is to treat yourself well. Enjoy the situation. You have only 1 life. So live happily, see the adversity you are facing as a challenge, as a problem solving opportunity. You just need to get 1 good chance, if you are smart and hardworking, your work will get paid off one day.
- Your algorithm is all correct but the input just keep on changing. You become a programmer with a lot of bugs.
- In a programming task, your decision was correct but your boss said no to your decision and insist his. You gain bad reputation in the process.
- You investigated a certain problem lower than the bottom. Your boss heard someone else without carefully considering your arguments. Not only you need to use a useless method to work on a problem you know how to work out again, you also waste the time of the first investigation.
- Your colleagues need your help. You help them. But your boss said you are not being putting your A-game into your tasks.
- At the end of the cycle, you were suddenly asked to implement brand new features which are all brainstormy in nature. There is no time to test them. If you don't do it, your job is at risk. If you do it, you are killing your colleagues.
- Your boss to use a college friend of his or a relative of a director. He asked you to "manage" the guy. If something good happens, the merit belong to that guy. If something bad happens, you need to be responsible.
The first thing you should do is to understand that this is not your fault. A pair of AA can get beat by Ax in Texas Holdem which every players know, it is a 93 to 7 shot. Sometimes you play well but you don't get pay well. That's just the way life is. Your boss, your peers as imperfect human beings, sometimes just make mistake. It is you to decide how to deal with this imperfect future.
The second thing is to make sure you still play well in the game. Don't go to drink because you are not happy for the day. Don't go to take drugs, smoke because you feel those are relieving things to do. They are counter-productive. Don't get emotional about these issues. If your boss or your colleague are obnoxious, eschew them. Make sure you make sure everything is correct in your domain. Control the outcome you can control.
The third thing to do is to keep on learning. There is always a better job waiting for you. Our discipline favor smart and hard-working people. There is always a better life outside the job you are working on. Every time you can get such a chance, you should ask for a salary adjustment. Why? Because we are hot. If you don't have a reason to stay in a place, leaving is not a bad idea. The best people in this country (also outside U.S.) move around. In that way, you can get benefit out of your jobs.
But remember, moving around different companies are not easy in our industry. You need to be good. So study, learn. Here the things I recommend you to learn:
1, Languages, Communications, Writing,
2, Math and basic computer science subjects. Top 10: Algorithms, Operating Systems, Compiler Design, Database, Discrete Mathematics, Probability and Random Process, Linear Algebra, Complexity Theory, Calculus, Machine Learning.
3, Management and Interpersonal Skill.
4, Musical Instruments,
5, Art,
6, Martial Arts.
1 and 2 are essential. All of them, when learn seriously, can open your mind. Being able to impress other people means a lot. 3 are accumulative. You just need to work and reflect some time. 4-6 will open your eyes.
The final thing to do is to treat yourself well. Enjoy the situation. You have only 1 life. So live happily, see the adversity you are facing as a challenge, as a problem solving opportunity. You just need to get 1 good chance, if you are smart and hardworking, your work will get paid off one day.
Sunday, May 29, 2011
My recent stuckness.
For the first time, I have come to a point I feel stuck when I try to proceed in CLRS. In my case, that means the three threads (browse read, exercise read and deep read) I have been working on are not moving.
Naturally, I am thinking about why. For each read, there are different reasons and I also figure out their solutions.
Naturally, I am thinking about why. For each read, there are different reasons and I also figure out their solutions.
- For my browse read, my understanding of tree, or generally my understanding in discrete mathematics, need to be improved quickly. Many of the questions require some intuitive understanding of trees. I will not open the daunting Rosen's Discrete Mathematics and Its Applications. Because I am fully aware of the damage of studying two text books at the same time. I will feel too burn out and feeling not learning anything. What I will do is to study Appendix B of CLRS and perhaps read the whole Chapter 2 of TAOCP. Then reread, chapter 12 and 13. This should prepare me better on the next reread.
- For my exercise read, as I mention before, I am beaten up by the exercises. My solution is to go to sleep and take some mental break from Algorithms In fact, I already line up couple of things to do while I am taking the mental break from algorithm. That includes memorizing 80 Kanjis and also read El Alquimista in Spanish. Both are within reach and fun to do.
- For my deep read, I just need to work on it one item by item. It is meant to be a deep connecting study to try to combine everything I know. This is a phase which takes time. I am just going to take my time and enjoy the process.
Saturday, May 28, 2011
Great Quote from Prof. Leiserson
I just watch on-line video for lecture 1 of the MIT open courseware in "Introduction to Algorithm". Great Lecture! and I think I learn a lot from it. It's bed time, so I will put down the best quote from Prof. Leiserson, one of the course lecturers.
"If you want to be a good program, you just program ever day for two years, you will be an excellent programmer. If you want to be a world-class programmer, you can program every day for ten years, or you can program every day for two years and take an algorithms class."
I am so glad I hear this. Though it's a bit late for me but I still feel it worths it.
"If you want to be a good program, you just program ever day for two years, you will be an excellent programmer. If you want to be a world-class programmer, you can program every day for ten years, or you can program every day for two years and take an algorithms class."
I am so glad I hear this. Though it's a bit late for me but I still feel it worths it.
Friday, May 27, 2011
Zen and the Art of Motorcycle Maintenance (Part 6)
Read another 10 pages. Now starting Chapter 26. This is indeed a difficult book to read.
Prisig suggested that one should unify both the subject and the object sides of quality together in a fundamental level. I don't know how this could be done but I know when this feeling appears to me.
Have you written a merge sort in the past? In the traditional top-down merge sort. There are usually three pieces of code: the merge function, the merge_sort function and the driver of the program.
When I look at individual piece, they look fairly ugly before you understand what they do as a whole.
After I finally got it running (,search merge_sort in this blog for my attempts), the feelings are entirely different. What a marvelous idea! In fact there is even an easy proof of its performance.
Then I went back to each of the program. Another feeling kicks in, wow, the program is indeed very smart. For a typical C implementation, you just need to pass in a piece of working memory. But the code does the magic. Unsorted array becomes sorted. How wonderful.
It is feeling like this keep us programmers working. Not money, not reputation, not fame. It is this beautiful feeling to get new things working make us keep on.
I guess we all have this feeling daily. It just a matter of recognizing it consciously.
Prisig suggested that one should unify both the subject and the object sides of quality together in a fundamental level. I don't know how this could be done but I know when this feeling appears to me.
Have you written a merge sort in the past? In the traditional top-down merge sort. There are usually three pieces of code: the merge function, the merge_sort function and the driver of the program.
When I look at individual piece, they look fairly ugly before you understand what they do as a whole.
After I finally got it running (,search merge_sort in this blog for my attempts), the feelings are entirely different. What a marvelous idea! In fact there is even an easy proof of its performance.
Then I went back to each of the program. Another feeling kicks in, wow, the program is indeed very smart. For a typical C implementation, you just need to pass in a piece of working memory. But the code does the magic. Unsorted array becomes sorted. How wonderful.
It is feeling like this keep us programmers working. Not money, not reputation, not fame. It is this beautiful feeling to get new things working make us keep on.
I guess we all have this feeling daily. It just a matter of recognizing it consciously.
Tuesday, May 24, 2011
Why I failed when I read TAOCP the first time.
Believe it or not, this is actually the second time I read TAOCP seriously. The first time is when I first came to U.S. (around 8 years ago) I was motivated by the success of reading "Elementary Number Theory". I finished all the exercises in the number theory book and felt very ambitious another book. I thought that I can gain a lot by reading TAOCP after I bought its international edition.
There's nothing wrong with that thinking. Unfortunately, I made mistakes which many people made, let me summarize at here.
There's nothing wrong with that thinking. Unfortunately, I made mistakes which many people made, let me summarize at here.
- I simply tried to read Chapter 1 too deep. TAOCP Chapter 1 gives a general introduction of computer science. This introduction is not gentle. In a way, it brings the reader to the frontier of each of these basic topics. (So if you go deep for TAOCP in one section, you are likely to be a deep experts in a field.) This is too difficult for me (or perhaps for many) to finish.
- I was scared by MIX. This is simply a bias. At that time I was crazily in love with C and has a lot of prejudice. I have since changed.
- I tried to read other CS text. Ah, one of my biggest problems, after reading something and have success, I liked to move on and go for breadth, than depth. Not a good habit for deep subject such as Math and CS.
- At that time, I wasn't proficient in basic programming skill. For example even though I finished a master already, I still have basic misunderstanding why writing good program is a good thing. Experience is my teacher.
- Just like many, I thought TAOCP was meant to be a reference. (Conform to the 99%, Big mistake.)
- I only think about algorithmic problem when I need to. (Conform to the 90%, Biggest mistake)
Sunday, May 22, 2011
Can CLRS be used as a self-study tool?
Absolutely.
Most of the time I heard people described books such as CLRS, Sedgewick and TAOCP are "reference books". If you are taking a class in CS, I guess that is an ok thinking. Though if you are a professional in computer programming, or if you need to change programs in a day-in day-out basis, I will say that CLRS, Sedgeick, TAOCP are completely for reading and for studying.
What confuses most of the people is that reading an algorithm book cannot be a cursory reading. i.e. what most of the people do most of the time. Reading an algorithmic book is meant to be as serious as reading a literature. You want to understand it in many levels. It can be sentence-by-sentence as you want to understand what the author means. It can also be an extraction of the semantic of the whole chapter, whole paragraphs. It can also be an understanding of the whole book. You may also need to read the book by reading many books in a certain subject to get an understanding.
Or more detail methods on how to read a book, I will suggest everyone to read Alder Mortimer, an Aristotelian's scholar, "How to read a book". There are many levels of reading described in his book. In his description, the ultimate level of reading would be syndicate read. That is to read many books and collate the combined information and gain a higher understanding.
When, many of us read, we basically subscribe to tenet of speed-reading in which we parse the text from left-to-right, wraparound and repeat as fast as possible. There is a place for speed-reading but books are not meant to be read in this way.
Algorithmic book has another requirement if you want to get something out of it. You need to read it with a computer and pieces of papers.
You need a computer because it is crucial to implement the algorithm to a certain extent to attain understanding of an algorithm. You may find that the algorithm described is wrong. That happens to books which are written by famous writers. You may find that you can do better and think of many variations of the algorithms. This will be prominent when you look up several algorithmic books. For example, there is not description of bottom up merge sort in CLRS, but there is one in Sedgewick. It's a useful skill to learn how to convert a recursive algorithm to an iterative one. So, learning both certainly don't hurt.
You also need papers because there are a lot of Math in algorithmic study. You want to understand a proof by possibly write it own. You want to think of variations of the proof. You want to extend the proof to a more general proof. (even if it is something you consider it as meaningless at the time.) Of course, you want to do exercises for the simpler books.
In fact, when you listen to very good programmers, you will find that they are always drilling on great algorithmic books. They know all the ins and outs of some simple algorithm. And if you listen to great mathematicians, they will tell you the best way to learn a subject is to try to prove all the theorems of a good standard text book on the field.
Everyone can be benefited from this way of learning. It is just a matter of a person's devotion, persistence and patience of the subject.
Most of the time I heard people described books such as CLRS, Sedgewick and TAOCP are "reference books". If you are taking a class in CS, I guess that is an ok thinking. Though if you are a professional in computer programming, or if you need to change programs in a day-in day-out basis, I will say that CLRS, Sedgeick, TAOCP are completely for reading and for studying.
What confuses most of the people is that reading an algorithm book cannot be a cursory reading. i.e. what most of the people do most of the time. Reading an algorithmic book is meant to be as serious as reading a literature. You want to understand it in many levels. It can be sentence-by-sentence as you want to understand what the author means. It can also be an extraction of the semantic of the whole chapter, whole paragraphs. It can also be an understanding of the whole book. You may also need to read the book by reading many books in a certain subject to get an understanding.
Or more detail methods on how to read a book, I will suggest everyone to read Alder Mortimer, an Aristotelian's scholar, "How to read a book". There are many levels of reading described in his book. In his description, the ultimate level of reading would be syndicate read. That is to read many books and collate the combined information and gain a higher understanding.
When, many of us read, we basically subscribe to tenet of speed-reading in which we parse the text from left-to-right, wraparound and repeat as fast as possible. There is a place for speed-reading but books are not meant to be read in this way.
Algorithmic book has another requirement if you want to get something out of it. You need to read it with a computer and pieces of papers.
You need a computer because it is crucial to implement the algorithm to a certain extent to attain understanding of an algorithm. You may find that the algorithm described is wrong. That happens to books which are written by famous writers. You may find that you can do better and think of many variations of the algorithms. This will be prominent when you look up several algorithmic books. For example, there is not description of bottom up merge sort in CLRS, but there is one in Sedgewick. It's a useful skill to learn how to convert a recursive algorithm to an iterative one. So, learning both certainly don't hurt.
You also need papers because there are a lot of Math in algorithmic study. You want to understand a proof by possibly write it own. You want to think of variations of the proof. You want to extend the proof to a more general proof. (even if it is something you consider it as meaningless at the time.) Of course, you want to do exercises for the simpler books.
In fact, when you listen to very good programmers, you will find that they are always drilling on great algorithmic books. They know all the ins and outs of some simple algorithm. And if you listen to great mathematicians, they will tell you the best way to learn a subject is to try to prove all the theorems of a good standard text book on the field.
Everyone can be benefited from this way of learning. It is just a matter of a person's devotion, persistence and patience of the subject.
Saturday, May 21, 2011
Zen and the Art of Motorcycle Maintenance (Part 2)
I finished another 30 pages. The book is so deep. It's obvious to me it is written by someone with great wisdom. Many issues such as the purpose of education system, the university system and our technologies were explained. This book opens my mind.
The feeling of not knowing something and obsessively want to understand run so deep to many technical people. Some of them never realize that in their whole lives. We just let time by to allow ourselves, for good and bad, not understand something. That's why I feel anxious. That's why I feel life is meaningless. That's why I feel all the sudden deep thought doesn't come to me. It is deep-seeded.
It is also when a person stopped to be in the flow. When does the flow stop?
Perhaps it starts when a person think that remembering a formula is equivalent to understanding. Or it could be the moment when we hear the phrase, hasta la vista, we can only think of Arnie. Or when we heard of Asian, we think solely of one race, one country. Or when we use a push in perl. Or the funny syntax in perl 5.0 which a one-liner can do both replacement and assign without understanding. (i.e. ($a = $b) =~ s/abc/xyz/g). Or feeling confused about obfuscated code. Or feeling frustrated when people named a new idea. Or feeling obliged to pay mortgage without knowing why we have mortgage. Frustrated by the economy without knowing what money is. Write the C-code, don't know what the compiler will do. Or write the Assembly code, but doesn't know how the machine takes it. Or know how the machine takes it, but doesn't know what it was designed this way.
We are immersed in systematically designed increasingly deepened abstraction. They are always shallow by definition. Only when you learn how it was created without applying the rules suggested by the others. Then you start to understand the nature.
That's why I am so frustrated because I don't understand all of these. More importantly, I or perhaps you misthought that not understanding them can solve any problems. It never does my friend. We are meant to know, to understand and finally come up with sublimed thought that conclude what we are thought and understood in our lives. Be it an idea, a paper, a research, a program, a company, something or anything.
The feeling of not knowing something and obsessively want to understand run so deep to many technical people. Some of them never realize that in their whole lives. We just let time by to allow ourselves, for good and bad, not understand something. That's why I feel anxious. That's why I feel life is meaningless. That's why I feel all the sudden deep thought doesn't come to me. It is deep-seeded.
It is also when a person stopped to be in the flow. When does the flow stop?
Perhaps it starts when a person think that remembering a formula is equivalent to understanding. Or it could be the moment when we hear the phrase, hasta la vista, we can only think of Arnie. Or when we heard of Asian, we think solely of one race, one country. Or when we use a push in perl. Or the funny syntax in perl 5.0 which a one-liner can do both replacement and assign without understanding. (i.e. ($a = $b) =~ s/abc/xyz/g). Or feeling confused about obfuscated code. Or feeling frustrated when people named a new idea. Or feeling obliged to pay mortgage without knowing why we have mortgage. Frustrated by the economy without knowing what money is. Write the C-code, don't know what the compiler will do. Or write the Assembly code, but doesn't know how the machine takes it. Or know how the machine takes it, but doesn't know what it was designed this way.
We are immersed in systematically designed increasingly deepened abstraction. They are always shallow by definition. Only when you learn how it was created without applying the rules suggested by the others. Then you start to understand the nature.
That's why I am so frustrated because I don't understand all of these. More importantly, I or perhaps you misthought that not understanding them can solve any problems. It never does my friend. We are meant to know, to understand and finally come up with sublimed thought that conclude what we are thought and understood in our lives. Be it an idea, a paper, a research, a program, a company, something or anything.
Thursday, May 19, 2011
Tire and Exhaustion
When you study solely on one single book. It can be a difficult feeling because you seldom really *complete* something. In my case, just finishing Chapter 2 makes me feel like I did something. But me, just like every one in the planet will feel having to finish the other 34 chapters are pretty darn daunting as well.
But I remind myself to keep on - the truth is I have been greatly benefited by this intensive studying.
I also remind myself this - in Math and CS, spending time upfront is the most important because material upfront are things you want to be excellent with. It's like doing the four simple operations of arithmetic. You can certainly skip them in young age but your number sense will be severely affected.
The lovely feeling of headache and feeling exhausted by Mathematical problems and brain teaser is back to me after many years. This time, no escape from it but simply just try to get benefited from it.
But I remind myself to keep on - the truth is I have been greatly benefited by this intensive studying.
I also remind myself this - in Math and CS, spending time upfront is the most important because material upfront are things you want to be excellent with. It's like doing the four simple operations of arithmetic. You can certainly skip them in young age but your number sense will be severely affected.
The lovely feeling of headache and feeling exhausted by Mathematical problems and brain teaser is back to me after many years. This time, no escape from it but simply just try to get benefited from it.
Tuesday, May 17, 2011
Why go deep in algorithm?
I had thought about this for long time and in fact try to experience what the best way is in my career.
It takes a long time to solve problems in any of the algorithm books. CLRS is perhaps the simplest. With the textbook and some college level of math, one can pretty much solve most of the problems and allow one to read on and on.
Difficulty of the problems aside, does it worth it? Does it worth it to go deep at all? If some algorithms will never being used, why are we learning them at all?
To me, the answer lies on the intelligent patterns which I learned from every algorithms and multiple perspectives offered from looking at each algorithms individually. It helps me a lot to solve new problems as well.
Besides, in real work, there are many beings closed to so called "script-kiddies" exist, what they do everyday is to program and patch the code from time to time if the original solution doesn't work. I was one of them. It doesn't seem me a smart thing to do and it give me great deal of embarrasment.
So I have no doubt a serious programmer should study at least one of the bibles - CLRS, Sedgewick or TAOCP. Preferably all of them but if one can learn from either one of them, I believe the programmer has genuinely upgraded.
The question is what is a reasonable amount to finish just one book? This is one thing I thought it through - if I decided to stay in the business which relates to computer, regardless of whether it is research, development, management or whatever. To make sure I made the best decision, data structure and algorithm is a must. So even if it takes 6 years to finish one book, I believe it's totally worth it. (Of course, it will be nice if I can somehow accelerate the process a bit.)
It takes a long time to solve problems in any of the algorithm books. CLRS is perhaps the simplest. With the textbook and some college level of math, one can pretty much solve most of the problems and allow one to read on and on.
Difficulty of the problems aside, does it worth it? Does it worth it to go deep at all? If some algorithms will never being used, why are we learning them at all?
To me, the answer lies on the intelligent patterns which I learned from every algorithms and multiple perspectives offered from looking at each algorithms individually. It helps me a lot to solve new problems as well.
Besides, in real work, there are many beings closed to so called "script-kiddies" exist, what they do everyday is to program and patch the code from time to time if the original solution doesn't work. I was one of them. It doesn't seem me a smart thing to do and it give me great deal of embarrasment.
So I have no doubt a serious programmer should study at least one of the bibles - CLRS, Sedgewick or TAOCP. Preferably all of them but if one can learn from either one of them, I believe the programmer has genuinely upgraded.
The question is what is a reasonable amount to finish just one book? This is one thing I thought it through - if I decided to stay in the business which relates to computer, regardless of whether it is research, development, management or whatever. To make sure I made the best decision, data structure and algorithm is a must. So even if it takes 6 years to finish one book, I believe it's totally worth it. (Of course, it will be nice if I can somehow accelerate the process a bit.)
Sunday, May 15, 2011
Jon Bentley's binary search: should we test a program before/after you write a program?
A must read of every programmer is the book Programming Pearl. If you care, you might have known that Programming Pearl's Chapter 4 and 5 have been a great reminder to every programmers that writing correct code is very hard. His example is the seemingly simple binary search. The concept is as simple as children's game : Twenty Questions. But to write it correctly, it's actually very hard. There are many correct variations of the algorithms which mainly lie on how the bounds were set (inclusive, exclusive) and termination condition. For anyone who has look down this algorithm, they will suffer the same of writing incorrect code.
Saying so, I got to admit that I am not such an exceptional programmer and I don't stand on higher moral ground. In fact, just like many algorithms I have practiced on, I need to write binary search for 4 times (including pseudocode and once in C) before I got it correct. In fact, not until I read Chapter 4 of Programming Pearl, I have no way to make sure my implementation (which at the end followed Bentley's implementation) is correct. This turns me from what Bentley said the 90% group of programmers who got binary search incorrect to the 10% who got the code correct.
Now here's the twist of the story, Joshua
This argument still has issues - one might argue that this is only true in this particular instance of programming. In other situations, what if there's another difference between the actual program language with the pseudo code? How do we know? My take is that this should be ok, because one can imagine that these differences of languages should have only finite number and one can enumerate them before they make a proof.
I think in this way because binary search gives a very good example where the testing tenet of programming doesn't work well. In the case if binary search was meant to search N numbers within range R : [p,q]? What would be the test case? The number of possible combination would be r ^N= (q-p+1) ^N (this allows repetition of the items). It is impossible to create test cases which cover the range of the program.
This exposed a fundamental issue of the whole tenet of programming by testing paradigm. Some extremist in the camp suggest that programming can be done by writing a program and then repeatedly test it until it generates the required I/O pairs. But in real life programming, there are just too many problems one can't handle in this way.
There is another argument against the testing paradigm which I consider it as a good practical reason but a bogus theoretical reason. This argument goes like this, even if one can test a program in the range R, what if we later decide that enlarge our range R to R'? If that's the case, then a program could fail because someone can simply store an R- entry table and because the new range of R' doesn't exist in the table, the program then fail.
As I said this is a bogus theoretical argument, because when one decided to enlarge R, then we are writing a different program. That doesn't mean the original R-entry table implementation is incorrect. It means that it is correct with the assumption R is the range.
In practice though, those who wield this argument will likely to have the last laugh because inputs/outputs of a particular function *always* change in real-life programming. No one can avoid it and no one can say no to it.
Saying so, I got to admit that I am not such an exceptional programmer and I don't stand on higher moral ground. In fact, just like many algorithms I have practiced on, I need to write binary search for 4 times (including pseudocode and once in C) before I got it correct. In fact, not until I read Chapter 4 of Programming Pearl, I have no way to make sure my implementation (which at the end followed Bentley's implementation) is correct. This turns me from what Bentley said the 90% group of programmers who got binary search incorrect to the 10% who got the code correct.
Now here's the twist of the story, Joshua
This argument still has issues - one might argue that this is only true in this particular instance of programming. In other situations, what if there's another difference between the actual program language with the pseudo code? How do we know? My take is that this should be ok, because one can imagine that these differences of languages should have only finite number and one can enumerate them before they make a proof.
I think in this way because binary search gives a very good example where the testing tenet of programming doesn't work well. In the case if binary search was meant to search N numbers within range R : [p,q]? What would be the test case? The number of possible combination would be r ^N= (q-p+1) ^N (this allows repetition of the items). It is impossible to create test cases which cover the range of the program.
This exposed a fundamental issue of the whole tenet of programming by testing paradigm. Some extremist in the camp suggest that programming can be done by writing a program and then repeatedly test it until it generates the required I/O pairs. But in real life programming, there are just too many problems one can't handle in this way.
There is another argument against the testing paradigm which I consider it as a good practical reason but a bogus theoretical reason. This argument goes like this, even if one can test a program in the range R, what if we later decide that enlarge our range R to R'? If that's the case, then a program could fail because someone can simply store an R- entry table and because the new range of R' doesn't exist in the table, the program then fail.
As I said this is a bogus theoretical argument, because when one decided to enlarge R, then we are writing a different program. That doesn't mean the original R-entry table implementation is incorrect. It means that it is correct with the assumption R is the range.
In practice though, those who wield this argument will likely to have the last laugh because inputs/outputs of a particular function *always* change in real-life programming. No one can avoid it and no one can say no to it.
Saturday, May 14, 2011
Good typer = Good coder?
After some thought, I don't think it is the case. Because coding requires precision of thinking as well. So that would explain why some slow typers are very good programmers. And some very good typers are very lousy programmers.
Mistakes made by a programmer - some hypotheses of why they are there.
I had a sense of disappointment after today's practice. It seems like I am still in the process of not able to write perfect code in the first shot. Of course I have improved but it makes you wonder how you can go beyond that.
After some thought, I believe this is an issue of concentration and how human write and check syntax of a statement. There are several issues here, one is the precision of my thought. To put the whole program in one's brain require more structure than I thought. For example, the seemingly simple insertion sort imply many constraints among the characters within its source code. So, every time you write an insertion sort, you don't just type an arbitrary stream of characters. You are typing characters which are interrelated with others in a certain way. Simply speaking that's syntax. But human brain is not a machine of syntax. That's why mistakes come up from time to time.
On that front, I believe it is an issue of training. Continuously put myself into the thinking process. Avoid fuzzy thinking. Always demand myself having a solid understanding of every tidbit of the algorithm and the environment. This will help.
The second issue is the how precise my hand can represented by thought. That is another layer of potential mistakes. For example, sometimes I think for(i=0;i<N;i++) but I then typed for(i=0;j<N;i++). Visually it doesn't look like a big mistake. It will also likely to skip checking.
For this I believe it is just an issue of professional typing skill. This can be improved by touch-typing training and can be improved if I desired to put time on it.
The third issue is checking. How come my mental checking fails for me? The reason is that my head doesn't always contain all the required syntax when I was checking my own words. How can I improve this? Hmm. I don't know. But I guess it is one type of human limitation which require special means to resolve.
Nevertheless, I am happy though that my self-study has expanded my horizon in algorithms. This is certainly something I can learn. It is also something that if I spent my time on it, I would improve. The most important of all, this is something if I spent my time on it, the payoff is handsome. That's why I am happy to put more time on it in future.
33_P
After some thought, I believe this is an issue of concentration and how human write and check syntax of a statement. There are several issues here, one is the precision of my thought. To put the whole program in one's brain require more structure than I thought. For example, the seemingly simple insertion sort imply many constraints among the characters within its source code. So, every time you write an insertion sort, you don't just type an arbitrary stream of characters. You are typing characters which are interrelated with others in a certain way. Simply speaking that's syntax. But human brain is not a machine of syntax. That's why mistakes come up from time to time.
On that front, I believe it is an issue of training. Continuously put myself into the thinking process. Avoid fuzzy thinking. Always demand myself having a solid understanding of every tidbit of the algorithm and the environment. This will help.
The second issue is the how precise my hand can represented by thought. That is another layer of potential mistakes. For example, sometimes I think for(i=0;i<N;i++) but I then typed for(i=0;j<N;i++). Visually it doesn't look like a big mistake. It will also likely to skip checking.
For this I believe it is just an issue of professional typing skill. This can be improved by touch-typing training and can be improved if I desired to put time on it.
The third issue is checking. How come my mental checking fails for me? The reason is that my head doesn't always contain all the required syntax when I was checking my own words. How can I improve this? Hmm. I don't know. But I guess it is one type of human limitation which require special means to resolve.
Nevertheless, I am happy though that my self-study has expanded my horizon in algorithms. This is certainly something I can learn. It is also something that if I spent my time on it, I would improve. The most important of all, this is something if I spent my time on it, the payoff is handsome. That's why I am happy to put more time on it in future.
33_P
Friday, May 13, 2011
Interview Questions
I just read a bunch of interview questions. The truth is programming has tons of trick questions and it can be daunting if you don't understand the basic of them. Three years ago when I was interviewed by a company, I was clearly overwhelmed by the techicalities required by interviewer.
Well, all are not lost, once I got solid fundamental from one famous books such as CLRS. It would be time to take up the challenge again. Before that, I just have to be patient and keep my mind sharp.
At least now, I have no problem if someone ask me algorithmic question. It is no longer something I would regard as almost impossible to understand. Almost everything has an answer, almost everything has a better answer. That's just the way it is. No need to feel emotional about it.
33_P
Well, all are not lost, once I got solid fundamental from one famous books such as CLRS. It would be time to take up the challenge again. Before that, I just have to be patient and keep my mind sharp.
At least now, I have no problem if someone ask me algorithmic question. It is no longer something I would regard as almost impossible to understand. Almost everything has an answer, almost everything has a better answer. That's just the way it is. No need to feel emotional about it.
33_P
Current reading methodologies for CLRS
What I am doing now is essentially a kind of multiple pass reading and deepening. Over the years, I believe this is the easiest way to implement for reading a long book. One of my test beds is poker books because the good ones usually take notoriously amount of time to understand.
My current strategy of reading CLRS split to 3 passes.
Read 1: Browse reading. Currently at Chapter 12. In this pass, I would try to go through the material as much as possible. The goal is to make sure I can keep on reading. A good book is usually structured in this way. The bad one will ask you to finish a certain exercise in order to read through. CLRS is one of those good books you can read through without relying on doing a lot of exercises. This read usually proceed very quickly. With my speed, we are talking about 1 chapter/day for simple chapters. Perhaps 2 days for difficult material such as binary tree or red-black tree.
Read 2: "Exercise" reading. Current up to P2.1. In this read, I attempt to finish all exercises for the book. For algorithmic questions. I will make sure I implement the solution in C. Just to keep myself skillful. This usually gives me the correct answer of a particular question. This takes some time to finish one exercise. But I usually learn a lot when I did.
Read 3: "Deep" reading. This will be the slowest and perhaps impossible to finish in a person's life time. But just figuring our one topic in one chapter is usually highly though provoking and can keep my mind sharp for a while. In this read, I will use all my imagination, all my curiosity to think of problems which is related to this topic. They might not have any answers per se. For example, in the case of insertion sort, I will want to learn all of its technicalities. All the questions human had asked about the algorithm. All the variants and all my ideas. The connections of it with other topics. This is also the moment where my own thought starts to come up.
Read 3 is the toughest and at this point I only go up to insertion sort of Chapter 2. All the things I want to think through will be recorded in complementary reading list which I updated constantly. This is probably the most important read but it's also highly dangerous as well. Because people who are curious are also easily distracted to other topics.
33_P
My current strategy of reading CLRS split to 3 passes.
Read 1: Browse reading. Currently at Chapter 12. In this pass, I would try to go through the material as much as possible. The goal is to make sure I can keep on reading. A good book is usually structured in this way. The bad one will ask you to finish a certain exercise in order to read through. CLRS is one of those good books you can read through without relying on doing a lot of exercises. This read usually proceed very quickly. With my speed, we are talking about 1 chapter/day for simple chapters. Perhaps 2 days for difficult material such as binary tree or red-black tree.
Read 2: "Exercise" reading. Current up to P2.1. In this read, I attempt to finish all exercises for the book. For algorithmic questions. I will make sure I implement the solution in C. Just to keep myself skillful. This usually gives me the correct answer of a particular question. This takes some time to finish one exercise. But I usually learn a lot when I did.
Read 3: "Deep" reading. This will be the slowest and perhaps impossible to finish in a person's life time. But just figuring our one topic in one chapter is usually highly though provoking and can keep my mind sharp for a while. In this read, I will use all my imagination, all my curiosity to think of problems which is related to this topic. They might not have any answers per se. For example, in the case of insertion sort, I will want to learn all of its technicalities. All the questions human had asked about the algorithm. All the variants and all my ideas. The connections of it with other topics. This is also the moment where my own thought starts to come up.
Read 3 is the toughest and at this point I only go up to insertion sort of Chapter 2. All the things I want to think through will be recorded in complementary reading list which I updated constantly. This is probably the most important read but it's also highly dangerous as well. Because people who are curious are also easily distracted to other topics.
33_P
Monday, May 9, 2011
Typing and Programming
When you have bad habit in typing. Suddenly fixing it will take a while to adapt to. But I decide to do it as slowly as I can. Such that I can learn again. I believe this should give me the biggest benefit.
It's just take more time. But it completely worths it.
It's just take more time. But it completely worths it.
Sunday, May 8, 2011
Money is never the ultimate goal
If you want to insult a programmer, keep him/her by paying double of his/her salary. Hint: the better programmer, you insult him/her more.
Money is never the ultimate goal
If you want to insult a programmer, keep him/her by paying double of his/her salary. Hint: the better programmer, you insult him/her more.
Why I am so frustrated with managers?
I believe the reason is that from a programmer's point of view. A manager shows consistency and logic. The role of manager also seems to make many seemingly brilliant fall from grace. A manager generally appears to resolve some issues which don't seem to make sense to me. Consider this, by default in a hierarchical work place. There is no guarantee for the manager to disclose what they are *really* thinking. There is no way for the subordinates to learn, to understand.
So a task which could be logical to a manager, can appear to a subordinate to be chaotic and irrational. From my experience, no one seems to be completely gotten out from this disease of human beings.
That's why I think humility is so important in the technical world. It is even more important in a technical workplace where hierarchy is imposed. Frustration was already a must. With an arrogant manager, this frustration is easily magnified. This could easily affect the whole team dynamics. If this issue persist, a whole technical organization can easily brought down.
And humility is so difficult to find. That's why I am so frustrated with manager. Especially when trust between me and the managers become thin.
So a task which could be logical to a manager, can appear to a subordinate to be chaotic and irrational. From my experience, no one seems to be completely gotten out from this disease of human beings.
That's why I think humility is so important in the technical world. It is even more important in a technical workplace where hierarchy is imposed. Frustration was already a must. With an arrogant manager, this frustration is easily magnified. This could easily affect the whole team dynamics. If this issue persist, a whole technical organization can easily brought down.
And humility is so difficult to find. That's why I am so frustrated with manager. Especially when trust between me and the managers become thin.
Subscribe to:
Posts (Atom)