The maximum entropy principle (Shannon (1948), Jaynes (1957)) provides a means to obtain least-biased statistical inference when insufficient information is available. Stemming from my prior work (NEM, polygonal FEM), the principle of maximum entropy was used to construct basis functions. The basis functions are viewed as a discrete probability distribution, and for n distinct nodes, the linear reproducing (precision) conditions are the constraints. For n > 3, the constraints represent an under-determined linear system. The maximum entropy variational principle is invoked, which leads to a unique solution with an exponential form for the basis functions. The maximum entropy approximant is valid for any point within the convex hull of the set of nodes (Sukumar, 2004), with interior nodal basis functions vanishing on the boundary of the convex hull (Fig. 1b). The use of variational principles (finite elements, conjugate gradient methods, graphical models, dynamic programming, statistical mechanics) is also appealing in data approximation (for example, Kriging, thin-plate splines, RBFs, MLS, Laplace, etc.).

In an independent study, Arroyo and Ortiz (2006) have shown the promise of local maximum entropy (convex) approximation schemes in a meshfree Galerkin method. Furthermore, links to convex analysis, extension to higher-order approximations, and the key properties of convex approximants are established. A general prescription of locally- and globally-supported convex approximation schemes can be derived using the Kullback-Leibler distance or directed divergence (principle of minimum relative entropy), which is presented in Sukumar (2005) and further elaborated in Sukumar and Wright (2007). The above studies share common elements with the Ph.D. thesis-research of Gupta (2003). For background on probability theory and Bayesian inductive inference, the books by Jaynes and Sivia are highly recommended.

Reports and Publications

• R. Silva-Valenzuela, A. Ortiz-Bernardin, N. Sukumar, E. Artioli and N. Hitschfeld-Kahler (2020), "A Nodal Integration Scheme for Meshfree Galerkin Methods using the Virtual Element Decomposition," International Journal for Numerical Methods in Engineering, Vol. 121, Number 10, pp. 2174–2205.
• A. Ortiz-Bernardin, A. Russo and N. Sukumar (2017), "Consistent and Stable Meshfree Galerkin Methods using the Virtual Element Decomposition," International Journal for Numerical Methods in Engineering, Vol. 112, Number 7, pp. 655–684.
• A. Ortiz-Bernardin, M. A. Puso and N. Sukumar (2015), "Improved Robustness for Nearly-Incompressible Large Deformation Meshfree Simulations on Delaunay Tessellations," Comput. Meth. Appl. Mech. Engrg., Vol. 293, pp. 348–374.
• D. Millán, N. Sukumar and M. Arroyo (2015), "Cell-Based Maximum-Entropy Approximants," Comput. Meth. Appl. Mech. Engrg., Vol. 284, pp. 712–731.
• N. Sukumar (2013), "Quadratic Maximum-Entropy Serendipity Shape Functions for Arbitrary Planar Polygons," Comput. Meth. Appl. Mech. Engrg., Vol. 263, pp. 27–41.
• F. Greco and N. Sukumar (2013), "Derivatives of Maximum-Entropy Basis Functions on the Boundary: Theory and Computations," International Journal for Numerical Methods in Engineering, Vol. 94, Number 12, pp. 1123–1149.
• G. Quaranta, S. K. Kunnath and N. Sukumar (2012), "Maximum-Entropy Meshfree Method for Nonlinear Static Analysis of Planar Reinforced Concrete Structures," Engineering Structures, Vol. 42, pp. 179–189.
• A. Ortiz, M. A. Puso and N. Sukumar (2011), "Maximum-Entropy Meshfree Method for Incompressible Media Problems," Finite Element in Analysis and Design, Vol. 47, Number 6, pp. 572–585.
• A. Ortiz, M. A. Puso and N. Sukumar (2010), "Maximum-Entropy Meshfree Method for Compressible and Near-Incompressible Elasticity," Comput. Meth. Appl. Mech. Engrg., Vol. 199, Number 25–28, pp. 1859–1871.
• L. L. Yaw, N. Sukumar and S. K. Kunnath (2009), "Meshfree Co-Rotational Formulation for Two-Dimensional Continua," International Journal for Numerical Methods in Engineering, Vol. 79, Number 8, pp. 979–1003.
• K. Hormann and N. Sukumar (2008), "Maximum Entropy Coordinates for Arbitrary Polytopes," Computer Graphics Forum, Vol. 27, Number 5, pp. 1513–1520. Proceedings of SGP 2008.
• N. Sukumar and R. J-B Wets (2007), "Deriving the Continuity of Maximum-Entropy Basis Functions via Variational Analysis," SIAM Journal of Optimization, Vol. 18, Number 3, pp. 914–925. or Journal
• N. Sukumar and R. W. Wright (2007), "Overview and Construction of Meshfree Basis Functions: From Moving Least Squares to Entropy Approximants," International Journal for Numerical Methods in Engineering, Vol. 70, Number 2, pp. 181–205.
• N. Sukumar (September 2006), "Where Do We Stand on Meshfree Approximation Schemes?" in Online Blog on Meshfree Methods or or
• N. Sukumar and E. A. Malsch (2006), "Recent Advances in the Construction of Polygonal Finite Element Interpolants," Archives of Computational Methods in Engineering, Vol. 13, Number 1, pp. 129–163. (proof)
• N. Sukumar (2005), "Maximum Entropy Approximation," in Proceedings of the 25th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, Eds. K. H. Knuth, A. E. Abbas, R. D. Morris and J. P. Castle, AIP Conference Proceedings, Vol. 803, Number 1, pp. 337–344.
• N. Sukumar (2004), "Construction of Polygonal Interpolants: A Maximum Entropy Approach," International Journal for Numerical Methods in Engineering, Vol. 61, Number 12, pp. 2159–2181

Presentations

• "Maximum-Entropy Approximation Schemes in Computational Mechanics," Invite Seminar, Sandia National Laboratories, Livermore, CA, September 2018.

• "Maximum-Entropy Approximation Schemes in Computational Mechanics," Plenary Lecture, USACM Thematic Workshop on Meshfree and Particle Methods: Application and Theory, Santa Fe, NM, September 2018.

• "Maximum-Entropy Approximation Schemes in Computational Mechanics," Invited Seminar, Dassault Systemes Simulia Corporation, Johnston, RI, April 2017.

• "Maximum-Entropy Approximation Schemes in Computational Mechanics," Invited Seminar, Pierce Laboratory Seminar Series, Department of Civil and Environmental Engineering, MIT, Cambridge, MA, April 2017.

• "Maximum-Entropy Approximation Schemes in Computational Mechanics," Invited Seminar, Joint Materials/Solid Mechanics Seminar, Brown University, Providence, RI, April 2017.

• "Maximum-Entropy Meshfree Methods in Computational Mechanics," Opening Lecture, 42nd Israeli Symposium on Computational Mechanics (ISCM-42), Technion, Haifa, Israel, March 2017.

• "Feasibility Constraints for Smooth Convex Data Approximations via Semidefinite Programming (with Jean B. Lasserre)," USACM Conference on Isogeometric Analysis and Meshfree Methods, San Diego, CA, October 2016.

• "Comparison of Meshfree Galerkin Methods Based on MLS and Maximum-Entropy Approximation Schemes (with Mathieu Foca and Bo Li)," ECCOMAS Thematic Conference: X-DMS 2015 - eXtended Discretization Methods, Ferrara, Italy, September 2015.

• "Recent Developments in Maximum-Entropy Approximation Schemes," Invited Seminar, Department of Engineering Design, IIT Madras, Chennai, India, June 2015.

• "Recent Developments in Maximum-Entropy Approximation Schemes," Invited Seminar, Department of Mechanical and Aerospace Engineering, Case Western Reserve University, Cleveland, OH, March 2015.

• "Cell-Based Maximum-Entropy Approximants (with Marino Arroyo and Daniel Millan)," Invited Lecture, Workshop on Meshfree Methods for Large-Scale Computational Science and Engineering Tampa, FL, October 2014.

• "Maximum-Entropy Approximation Schemes: Theory and Links to B-Splines and Isogeometric Analysis (with Marino Arroyo)," Isogeometric Analysis: Integrating Design and Analysis (IGA 2014), Austin, TX, January 2014.

• "Quadratic Maximum-Entropy Serendipity Shape Functions for Arbitrary Planar Polygons," Invited Lecture, 1st Indian Conference on Applied Mechanics, IIT Madras, Chennai, India, July 2013.

• "Maximum-Entropy Approximation Schemes," Invited Seminar, Department of Mechanical Engineering, Indian Institute of Science, Bangalore, India, June 2013.

• "Meshfree Methods: From the Element-Free Galerkin Method to Maximum-Entropy Approximation Schemes," Invited Lecture, Advances in Computational Mechanics with Emphasis on Fracture and Multiscale Phenomena: Workshop Honoring Professor Ted Belytschko's 70th Birthday, Evanston, IL, April 2013.

• "Quadratic Maximum-Entropy Serendipity Shape Functions on Arbitrary Planar Polygons," Invited Lecture, Minisymposium on Meshfree and Generalized Finite Element Methods, Advances in Computational Mechanics: A Conference Celebrating the 70th Birthday of Thomas J. R. Hughes, San Diego, February 2013.

• "Generalized Barycentric Coordinates on Polygons and Polyhedra," Invited Seminar, Combined Applied Math and PDEs Seminar, Department of Mathematics, University of California at Davis, October 2012.

• "Quadratic Polygonal Finite Element Methods," Invited Seminar, Faculty of Informatics, Universita della Svizzera italiana, Lugano, Switzerland. September 21, 2012.

• "Polygonal Finite Element Methods," Invited Lecture, Workshop on Discretization Methods for Polygonal and Polyhedral Meshes, University of Milano-Bicocca, Milan, Italy. September 17-19, 2012.

• "Barycentric Finite Element Methods," Invited Lecture, Workshop on Generalized Barycentric Coordinates in Geometry Processing and Finite/Boundary Element Methods, Columbia University, New York, July 26, 2012.

• "Maximum-Entropy Approximants: Applications in Solid Mechanics and Structural Engineering," Invited Seminar, Department of Civil and Environmental Engineering, University of Pittsburgh, Pittsburgh, PA, February 2012.

• "Maximum-Entropy Approximation Schemes," Symposium on Meshfree and Extended/ Generalized Finite Element Methods, PACAM XII, Port of Spain, Trinidad, January 2012.

• "Maximum-Entropy Approximations in Computational Mechanics," Invited Seminar, Department of Applied Mechanics, IIT Madras, Chennai, India, December 2011.

• "Maximum-Entropy Meshfree Method in Computational Mechanics," Invited Seminar, Universidad Politecnica de Valencia, Spain, July 2010.

• "Maximum-Entropy Meshfree Method," Invited Seminar, Student-Run Applied & Math Seminar, Department of Mathematics, University of California, Davis, CA, April 2010.

• "Maximum-Entropy Approximation in Computational Mechanics," Invited Seminar, Department of Civil and Environmental Engineering, Structural Engineering Seminar Series, University of Illinois at Urbana-Champaign, October 2009.

• "Maximum Entropy Coordinates for Arbitrary Polytopes (with Kai Hormann)," 10th US National Congress on Computational Mechanics, Symposium on Synergies in Computational Mechanics and Geometry, Columbus, OH, July 2009.

• "Barycentric Finite Element Methods," Invited Lecture, Symposium on Barycentric Coordinates and Transfinite Interpolation, Tenth SIAM Conference on Geometric Design & Computing, San Antonio, TX, November 5, 2007.

• "Linking Geometry and Approximation for Computational Mechanics," Mechanics Colloquim, Technical University of Kaiserslautern, Germany, September 2006

• "Continuity of Maximum-Entropy Basis Functions via Variational Analysis (with R. J-B Wets)," Seventh World Congress on Computational Mechanics, Symposium on Computational Geometry and Analysis, Los Angeles, CA, July 2006.

• "Construction of Meshfree Approximation Schemes: Use of Information-Theoretic Variational Principles," Invited Seminar, Technical University of Kaiserslautern, Germany, September 2005

• "Maximum Entropy Approximation," 25th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, San Jose State University, CA, August 2005

• "Maximum Entropy Approximation," Symposium on Meshfree and Particle Methods, Keynote Lecture, Eight U.S. National Congress on Computational Mechanics, Austin, TX, July 2005

• "Scattered Data Approximation using the Maximum Entropy Principle," Invited Seminar, Department of Civil and Environmental Engineering, Cornell University, Ithaca, NY, May 2005; Invited Seminar, Department of Mechanical and Aeronautical Engineering, University of California, Davis, CA, December 2004.

• MAXENT-F90: Fortran 90 library for maximum-entropy basis functions

• Matlab code and C++ meshfree class

• Basis functions in one dimension: [MLS, MAXENT, MIN RELATIVE ENTROPY]
  (For instructions, type `help mls1d', `help maxent1d', and `help relent1d' within Matlab)

• Matlab routines for maximum-entropy approximants (by Marino Arroyo)

• JAVA applets to visualize meshfree basis functions: [1D, 2D, 3D]
  (For instructions on using the applets, scroll down the web page in the 2D and 3D applets)

• ECI 216: Meshfree and Partition of Unity Methods (Fall 2018, Spring 2024)

Disclaimer: Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation