The maximum entropy principle (Shannon (1948), Jaynes (1957)) provides a means to obtain least-biased statistical inference when insufficient information is available. Stemming from my prior work (NEM, polygonal FEM), the principle of maximum entropy was used to construct basis functions. The basis functions are viewed as a discrete probability distribution, and for n distinct nodes, the linear reproducing (precision) conditions are the constraints. For n > 3, the constraints represent an under-determined linear system. The maximum entropy variational principle is invoked, which leads to a unique solution with an exponential form for the basis functions. The maximum entropy approximant is valid for any point within the convex hull of the set of nodes (Sukumar, 2004), with interior nodal basis functions vanishing on the boundary of the convex hull (Fig. 1b). The use of variational principles (finite elements, conjugate gradient methods, graphical models, dynamic programming, statistical mechanics) is also appealing in data approximation (for example, Kriging, thin-plate splines, RBFs, MLS, Laplace, etc.).

In an independent study, Arroyo and Ortiz (2006) have shown the promise of local maximum entropy (convex) approximation schemes in a meshfree Galerkin method. Furthermore, links to convex analysis, extension to higher-order approximations, and the key properties of convex approximants are established. A general prescription of locally- and globally-supported convex approximation schemes can be derived using the Kullback-Leibler distance or directed divergence (principle of minimum relative entropy), which is presented in Sukumar (2005) and further elaborated in Sukumar and Wright (2007). The above studies share common elements with the Ph.D. thesis-research of Gupta (2003). For background on probability theory and Bayesian inductive inference, the books by Jaynes and Sivia are highly recommended.

Reports and Publications

• K. Hormann and N. Sukumar (2008), "Maximum Entropy Coordinates for Arbitrary Polytopes," Computer Graphics Forum, Vol. 27, Number 5, pp. 1513–1520. Proceedings of SGP 2008.
• N. Sukumar and R. J-B Wets (2007), "Deriving the Continuity of Maximum-Entropy Basis Functions via Variational Analysis," SIAM Journal of Optimization, Vol. 18, Number 3, pp. 914–925. or Journal
• N. Sukumar and R. W. Wright (2007), "Overview and Construction of Meshfree Basis Functions: From Moving Least Squares to Entropy Approximants," International Journal for Numerical Methods in Engineering, Vol. 70, Number 2, pp. 181–205.
• N. Sukumar (September 2006), "Where Do We Stand on Meshfree Approximation Schemes?" in Online Blog on Meshfree Methods or
• N. Sukumar and E. A. Malsch (2006), "Recent Advances in the Construction of Polygonal Finite Element Interpolants," Archives of Computational Methods in Engineering, Vol. 13, Number 1, pp. 129–163. (proof)
• N. Sukumar (2005), "Maximum Entropy Approximation," in Proceedings of the 25th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Eds. K. H. Knuth, A. E. Abbas, R. D. Morris and J. P. Castle, AIP Conference Proceedings, Vol. 803, Number 1, pp. 337–344.
• N. Sukumar (2004), "Construction of Polygonal Interpolants: A Maximum Entropy Approach," International Journal for Numerical Methods in Engineering, Vol. 61, Number 12, pp. 2159–2181

Presentations

• "Barycentric Finite Element Methods," Invited Lecture, Symposium on Barycentric Coordinates and Transfinite Interpolation, Tenth SIAM Conference on Geometric Design & Computing, San Antonio, TX, November 5, 2007.

• "Linking Geometry and Approximation for Computational Mechanics," Mechanics Colloquim, Technical University of Kaiserslautern, Germany, September 2006

• "Continuity of Maximum-Entropy Basis Functions via Variational Analysis (with R. J-B Wets)," Seventh World Congress on Computational Mechanics, Symposium on Computational Geometry and Analysis, Los Angeles, CA, July 2006.

• "Construction of Meshfree Approximation Schemes: Use of Information-Theoretic Variational Principles," Invited Lecture, Technical University of Kaiserslautern, Germany, September 2005

• "Maximum Entropy Approximation," 25th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, San Jose State University, CA, August 2005

• "Maximum Entropy Approximation," Symposium on Meshfree and Particle Methods, Keynote Lecture, Eight U.S. National Congress on Computational Mechanics, Austin, TX, July 2005

• "Scattered Data Approximation using the Maximum Entropy Principle," Invited Lecture, Department of Civil and Environmental Engineering, Cornell University, Ithaca, NY, May 2005; Invited Lecture, Department of Mechanical and Aeronautical Engineering, University of California, Davis, CA, December 2004.

• Fortran 90 library for maximum-entropy basis functions

• Basis functions in one dimension: [MLS, MAXENT, MIN RELATIVE ENTROPY]
  (For instructions, type `help mls1d', `help maxent1d', and `help relent1d' within Matlab)

• Matlab routines for maximum-entropy approximants (by Marino Arroyo)

• JAVA applets to visualize meshfree basis functions: [1D, 2D, 3D]
  (For instructions on using the applets, scroll down the web page in the 2D and 3D applets)

Disclaimer: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation

Counter statistics available at to this page since Jan 20, 2006.