Bernard Widrow

All Publications

• The No-Prop algorithm: a new learning algorithm for multilayer neural networks. Neural networks Widrow, B., Greenblatt, A., Kim, Y., Park, D. 2013; 37: 182-188

  Abstract

  A new learning algorithm for multilayer neural networks that we have named No-Propagation (No-Prop) is hereby introduced. With this algorithm, the weights of the hidden-layer neurons are set and fixed with random values. Only the weights of the output-layer neurons are trained, using steepest descent to minimize mean square error, with the LMS algorithm of Widrow and Hoff. The purpose of introducing nonlinearity with the hidden layers is examined from the point of view of Least Mean Square Error Capacity (LMS Capacity), which is defined as the maximum number of distinct patterns that can be trained into the network with zero error. This is shown to be equal to the number of weights of each of the output-layer neurons. The No-Prop algorithm and the Back-Prop algorithm are compared. Our experience with No-Prop is limited, but from the several examples presented here, it seems that the performance regarding training and generalization of both algorithms is essentially the same when the number of training patterns is less than or equal to LMS Capacity. When the number of training patterns exceeds Capacity, Back-Prop is generally the better performer. But equivalent performance can be obtained with No-Prop by increasing the network Capacity by increasing the number of neurons in the hidden layer that drives the output layer. The No-Prop algorithm is much simpler and easier to implement than Back-Prop. Also, it converges much faster. It is too early to definitively say where to use one or the other of these algorithms. This is still a work in progress.

  View details for DOI 10.1016/j.neunet.2012.09.020

  View details for PubMedID 23140797

• A unified framework for 3D radiation therapy and IMRT planning: plan optimization in the beamlet domain by constraining or regularizing the fluence map variations PHYSICS IN MEDICINE AND BIOLOGY Meng, B., Zhu, L., Widrow, B., Boyd, S., Xing, L. 2010; 55 (22): N521-N531

  Abstract

  The purpose of this work is to demonstrate that physical constraints on fluence gradients in 3D radiation therapy (RT) planning can be incorporated into beamlet optimization explicitly by direct constraint on the spatial variation of the fluence maps or implicitly by using total-variation regularization (TVR). The former method forces the fluence to vary in accordance with the known form of a wedged field and latter encourages the fluence to take the known form of the wedged field by requiring the derivatives of the fluence maps to be piece-wise constant. The performances of the proposed methods are evaluated by using a brain cancer case and a head and neck case. It is found that both approaches are capable of providing clinically sensible 3D RT solutions with monotonically varying fluence maps. For currently available 3D RT delivery schemes based on the use of customized physical or dynamic wedges, constrained optimization seems to be more useful because the optimized fields are directly deliverable. Working in the beamlet domain provides a natural way to model the spatial variation of the beam fluence. The proposed methods take advantage of the fact that 3D RT is a special form of intensity-modulated radiation therapy (IMRT) and finds the optimal plan by searching for fields with a certain type of spatial variation. The approach provides a unified framework for 3D CRT and IMRT plan optimization.

  View details for DOI 10.1088/0031-9155/55/22/N01

  View details for Web of Science ID 000283789700001

  View details for PubMedID 21030744

• Adaptive Cancellation of Floor Vibrations in Standing Ballistocardiogram Measurements Using a Seismic Sensor as a Noise Reference IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING Inan, O. T., Etemadi, M., Widrow, B., Kovacs, G. T. 2010; 57 (3): 722-727

  Abstract

  An adaptive noise canceller was used to reduce the effect of floor vibrations on ballistocardiogram (BCG) measurements from a modified electronic bathroom scale. A seismic sensor was placed next to the scale on the floor and used as the noise reference input to the noise canceller. BCG recordings were acquired from a healthy subject while another person stomped around the scale, thus causing increased floor vibrations. The noise canceller substantially eliminated the artifacts in the BCG signal due to these vibrations without distorting the morphology of the measured BCG. Additionally, recordings were obtained from another subject standing inside a parked bus while the engine was running. The artifacts due to the vibrations of the engine, and the other vehicles moving on the road next to the bus, were also effectively eliminated by the noise canceller. The system with automatic floor vibration cancellation could be used to increase BCG measurement robustness in home monitoring applications. Additionally, the noise cancellation approach may enable BCG recording in ambulances-or other transport vehicles-where noninvasive hemodynamic monitoring may otherwise not be feasible.

  View details for DOI 10.1109/TBME.2009.2018831

  View details for Web of Science ID 000274990800025

  View details for PubMedID 19362900

• Adaptive Cancellation of Floor Vibrations in Standing Ballistocardiogram Measurements Using a Seismic Sensor as a Noise Reference IEEE Transactions on Biomedical Engineering Widrow, B., Inan, O., Etemadi, M., Kovacs, G. 2010; 57 (3): 722-727
• Predicting respiratory tumor motion with multi-dimensional adaptive filters and support vector regression PHYSICS IN MEDICINE AND BIOLOGY Riaz, N., Shanker, P., Wiersma, R., Gudmundsson, O., Mao, W., Widrow, B., Xing, L. 2009; 54 (19): 5735-5748

  Abstract

  Intra-fraction tumor tracking methods can improve radiation delivery during radiotherapy sessions. Image acquisition for tumor tracking and subsequent adjustment of the treatment beam with gating or beam tracking introduces time latency and necessitates predicting the future position of the tumor. This study evaluates the use of multi-dimensional linear adaptive filters and support vector regression to predict the motion of lung tumors tracked at 30 Hz. We expand on the prior work of other groups who have looked at adaptive filters by using a general framework of a multiple-input single-output (MISO) adaptive system that uses multiple correlated signals to predict the motion of a tumor. We compare the performance of these two novel methods to conventional methods like linear regression and single-input, single-output adaptive filters. At 400 ms latency the average root-mean-square-errors (RMSEs) for the 14 treatment sessions studied using no prediction, linear regression, single-output adaptive filter, MISO and support vector regression are 2.58, 1.60, 1.58, 1.71 and 1.26 mm, respectively. At 1 s, the RMSEs are 4.40, 2.61, 3.34, 2.66 and 1.93 mm, respectively. We find that support vector regression most accurately predicts the future tumor position of the methods studied and can provide a RMSE of less than 2 mm at 1 s latency. Also, a multi-dimensional adaptive filter framework provides improved performance over single-dimension adaptive filters. Work is underway to combine these two frameworks to improve performance.

  View details for DOI 10.1088/0031-9155/54/19/005

  View details for Web of Science ID 000270051600006

  View details for PubMedID 19729711

• Quantization Noise: Round Off Error in Digital Computation, Signal Processing, Control and Communications Widrow, B., Kollar, I. Cambridge University Press. 2008
• Statistical efficiency of adaptive algorithms NEURAL NETWORKS Widrow, B., Kamenetsky, M. 2003; 16 (5-6): 735-744

  Abstract

  The statistical efficiency of a learning algorithm applied to the adaptation of a given set of variable weights is defined as the ratio of the quality of the converged solution to the amount of data used in training the weights. Statistical efficiency is computed by averaging over an ensemble of learning experiences. A high quality solution is very close to optimal, while a low quality solution corresponds to noisy weights and less than optimal performance. In this work, two gradient descent adaptive algorithms are compared, the LMS algorithm and the LMS/Newton algorithm. LMS is simple and practical, and is used in many applications worldwide. LMS/Newton is based on Newton's method and the LMS algorithm. LMS/Newton is optimal in the least squares sense. It maximizes the quality of its adaptive solution while minimizing the use of training data. Many least squares adaptive algorithms have been devised over the years, but no other least squares algorithm can give better performance, on average, than LMS/Newton. LMS is easily implemented, but LMS/Newton, although of great mathematical interest, cannot be implemented in most practical applications. Because of its optimality, LMS/Newton serves as a benchmark for all least squares adaptive algorithms. The performances of LMS and LMS/Newton are compared, and it is found that under many circumstances, both algorithms provide equal performance. For example, when both algorithms are tested with statistically nonstationary input signals, their average performances are equal. When adapting with stationary input signals and with random initial conditions, their respective learning times are on average equal. However, under worst-case initial conditions, the learning time of LMS can be much greater than that of LMS/Newton, and this is the principal disadvantage of the LMS algorithm. But the strong points of LMS are ease of implementation and optimal performance under important practical conditions. For these reasons, the LMS algorithm has enjoyed very widespread application. It is used in almost every modem for channel equalization and echo cancelling. Furthermore, it is related to the famous backpropagation algorithm used for training neural networks.

  View details for DOI 10.1016/S0893-6080(03)00126-6

  View details for Web of Science ID 000184011900028

  View details for PubMedID 12850029

• Least-Mean-Square Adaptive Filters Widrow, B., Haykin, S. Wiley-Interscience. 2003
• Statistical Efficiency of Adaptive Algorithms Neural Networks Widrow, B., Kamenetsky, M. 2003: 735-744
• Neurointerfaces IEEE Transactions on Control Systems Technology Widrow, B. 2002: 221-228
• Neural dynamic optimization for control systems - Part II: Theory IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART B-CYBERNETICS Seong, C. Y., Widrow, B. 2001; 31 (4): 490-501

  Abstract

  The paper presents neural dynamic optimization (NDO) as a method of optimal feedback control for nonlinear multi-input-multi-output (MIMO) systems. The main feature of NDO is that it enables neural networks to approximate the optimal feedback solution whose existence dynamic programming (DP) justifies, thereby reducing the complexities of computation and storage problems of the classical methods such as DP. This paper mainly describes the theory of NDO, while the two other companion papers of this topic explain the background for the development of NDO and demonstrate the method with several applications including control of autonomous vehicles and of a robot arm, respectively.

  View details for Web of Science ID 000170320400002

  View details for PubMedID 18244816

• Neural dynamic optimization for control systems - Part III: Applications IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART B-CYBERNETICS Seong, C. Y., Widrow, B. 2001; 31 (4): 502-513

  Abstract

  For pt.II. see ibid., p. 490-501. The paper presents neural dynamic optimization (NDO) as a method of optimal feedback control for nonlinear multi-input-multi-output (MIMO) systems. The main feature of NDO is that it enables neural networks to approximate the optimal feedback solution whose existence dynamic programming (DP) justifies, thereby reducing the complexities of computation and storage problems of the classical methods such as DP. This paper demonstrates NDO with several applications including control of autonomous vehicles and of a robot-arm, while the two other companion papers of this topic describes the background for the development of NDO and present the theory of the method, respectively.

  View details for Web of Science ID 000170320400003

  View details for PubMedID 18244817

• Neural dynamic optimization for control systems - Part I: Background IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS PART B-CYBERNETICS Seong, C. Y., Widrow, B. 2001; 31 (4): 482-489

  Abstract

  The paper presents neural dynamic optimization (NDO) as a method of optimal feedback control for nonlinear multi-input-multi-output (MIMO) systems. The main feature of NDO is that it enables neural networks to approximate the optimal feedback solution whose existence dynamic programming (DP) justifies, thereby reducing the complexities of computation and storage problems of the classical methods such as DP. This paper mainly describes the background and motivations for the development of NDO, while the two other subsequent papers of this topic present the theory of NDO and demonstrate the method with several applications including control of autonomous vehicles and of a robot arm, respectively.

  View details for Web of Science ID 000170320400001

  View details for PubMedID 18244815

• A Microphone Array for Hearing Aids IEEE Circuits and Systems Magazine Widrow, B. 2001: 26-32
• Adaptive inverse control based on linear and nonlinear adaptive filtering INTERNATIONAL WORKSHOP ON NEURAL NETWORKS FOR IDENTIFICATION, CONTROL, ROBOTICS, AND SIGNAL/IMAGE PROCESSING - PROCEEDINGS Widrow, B., Plett, G. L. 1996: 30-38

  View details for Web of Science ID A1996BG16U00004

• Nonlinear Control with Neural Networks Backpropagation: Theory, Architectures, and Applications Chauvin, Y., D. Rumelhart, D. Erlbaum Associates. 1995
• Noise Canceling and Channel Equalization Handbook of Brain Theory and Neural Networks Widrow, B., Lehr, M. MIT Press. 1995
• 30 Years of Adaptive Neural Networks: Perceptron, Madaline, and Backpropagation Neural Networks: Theoretical Foundations and Analysis Lau, C. IEEE Press. 1992
• 30 Years of Adaptive Neural Networks: Perceptron, Madaline, and Backpropagation Artificial Neural Networks: Paradigms, Applications, and Hardware Implemenation Sanchez-Sinencio, E., Lau, C. IEEE Press. 1992: 82-108
• Sensitivity of feedforward neural networks to weight errors. IEEE transactions on neural networks Stevenson, M., Winter, R., Widrow, B. 1990; 1 (1): 71-80

  Abstract

  An analysis is made of the sensitivity of feedforward layered networks of Adaline elements (threshold logic units) to weight errors. An approximation is derived which expresses the probability of error for an output neuron of a large network (a network with many neurons per layer) as a function of the percentage change in the weights. As would be expected, the probability of error increases with the number of layers in the network and with the percentage change in the weights. The probability of error is essentially independent of the number of weights per neuron and of the number of neurons per layer, as long as these numbers are large (on the order of 100 or more).

  View details for PubMedID 18282824

• 30 Years of Adaptive Neural Networks: Perceptron, Madaline, and Backpropagation Widrow, B., Lehr, M. 1990: 1415-1442
• Fundamental Relations Between the LMS Algorithm and the DFT IEEE Transactions on Circuits and Systems Widrow, B., Baudrenghien, P., Vetterli, M., Titchener, P. 1987: 814-820
• Adaptive Signal Processing Widrow, B., Stearns, S. Prentice Hall. 1985
• On the Statistical Efficiency of the LMS Algorithm with Nonstationary Inputs IEEE Transactions on Information Theory Widrow, B., Walach E. 1984: 211-221
• Adaptive Filters Aspects of Network and System Theory Kalman, R., DeClaris, N. Holt, Rinehart and Winston. 1971
• Adaptive Antenna Systems [a citation classic] Widrow, B., Mantey, P. 1967: 2143-2159
• Statistical Analysis of Amplitude-Quantized Sampled-Data Systems AIEE Transactions on Applications and Industry Widrow, B. 1961: 1-14
• Adaptive Switching Circuits RE WESCON Convention Record Widrow, B. 1960: 96-104
• A Study of Rough Amplitude Quantization by Means of Nyquist Sampling Theory IRE Transactions on Circuit Theory Widrow, B. 1956; CT-3(4): 266-276