GNU Octave is a high-level language, primarily intended for numerical computations. It provides a convenient command line interface for solving linear and nonlinear problems numerically, and for performing other numerical experiments using a language that is mostly compatible with Matlab.
Octave is an open source with great tools. The commands are identical to those of MatLab.
Octave.app for Mac OS X is a ready to run binary version of GNU Octave. It runs on most of Apple's Mac OS X computers and comes with all the free libraries that are needed. The application can also be expanded by downloading and installing further packages as described before.
Documentation: there are several good tutorials out there: