When you program a computer, you make it process symbols in such a way that the results mean something to you. Whether the symbols and results mean anything to the computer is not a consideration to programmers, nor is it an easy philosophical question. Yet it is important because computers cannot catch semantic errors unless they know the meanings of symbols. In the Robot Baby project we have robots learn the meanings of symbols. I will describe work on learning the meanings of words in speech and text, and the meanings of states in planning. As a class, the learning algorithms find structure in multivariate time series, and they are generally useful in data mining from sequences and series. We have been interested to see that different aspects of meaning are extracted by different algorithms. I will conclude by describing two challenge problems that have valuable applications and also drive research in the direction of semantic autonomy for agents.