The EKF has been widely accepted as a standard tool in the machine learning community. In this paper we have presented an alternative to the EKF using the unscented filter. The UKF consistently achieves a better level of accuracy than the EKF at a comparable level of complexity. We have demonstrated this performance gain in a number of application domains, including state-estimation, dual estimation, and parameter estimation. Future work includes additional characterization of performance benefits, extensions to batch learning and non-MSE cost functions, as well as application to other neural and non-neural ( e.g., parametric) architectures. In addition, we are also exploring the use of the UKF as a method to improve Particle Filters [12], as well as an extension of the UKF itself that avoids the linear update assumption by using a direct Bayesian update [13].