Effective Minimization of Ill-conditioned Functions

The problem of minimization of ill-conditioned functions is considered. These functions are motivated by the limit analysis problem for dielectrics in powerful electric fields. From computing point of view a minimization of these functions is not following by standard methods because they are nonsmooth. Distinct ravine of objective function in combination with a high dimension variables requires a smooth regularisation of finite-dimensional problem. Convergence of heave-ball method in relation to it’s internal parameters and optimization parameters are studied in numerical computing environment and fourth-generation programming language Matlab.


Introduction
Investigation of electrical strength of dielectrics particular interest in both theory and practice [1]. It is stimulated by significance and practical interests in field of electrical engineering and microelectronics. When dielectrics are located in powerful electric fields it appears there the effect of loss of global electrostatic balance state between external electric field and dielectric. This effect we interpret as electrical puncture of dielectric [3][4][5][6][7].
The electrical state of a medium in the given domain is is the electrical constant, and P is the vector of polarization. For the electrical tensity the scalar potential u is introduced such as u E −∇ = . In weak electrical fields the currents of conductivity in dielectric media are practically absent, i.e. 0 ≈ J , and the simplest linear constitutive relation D E → is used [1]. As a result, for the solution of the appropriate linear BVPs, various effective analytical and numerical methods have been worked out.

The Limit Analysis Problem in Electrostatics
In powerful electrical fields the essentially nonlinear phenomena of polarization saturation ( ) +∞ ≤ < * P P and ionization ( ) 0 ≠ J must be taken into account [1]. As a result, the integral model of bounded electrical induction is used, where +∞ ≤ < λ D and 0 > λ is the complementary physical parameter of dielectric medium (the parameter of saturation) which is easily calculated [3][4][5][6][7].
From the physical point of view this effect is interpreted as loss of the global electrostatic balance state between external electric field and dielectric on the analogy of loss of the load-carrying capability of solids in mechanics [7].
Let a dielectric medium occupy a domain 3 R ∈ Ω . It's described by two vector-functions ( ) We consider the following boundary-value problem. The quasi-static electrical influences acting on the dielectric are: a bulk charge with density ρ in Ω , a surface charge with density g on a portion 2 Г of the boundary, and a potential 0 u on a portion 1 Г of the boundary is also given. Here and area(Г 1 )>0. Point charges absent.
We assume that from the classical Thomson's principle [1] it follows that the free energy of the electrical field in dielectric has the global minimum on the real potential, i.e. the potential u is the solution of the following variational problem: is the set of admissible electrical potentials, A(u) is the work of the electrical field on the external charges, Ф is the specific full energy of the electrical field such that for every 3 R E ∈ and almost every Ω ∈ x .
In the general case, according to the Thomson law and Joule-Lenz law specific potential energy of electric-field with regard to noninvertible losses from the current of conductivity is searched by formula: where point is the scalar product in R 3 . In the work [3] it is shown that there is always exists such parameter of electrostatic saturation λ>0, that for any 3

R E ∈
and almost all Ω ∈ x an estimation is faithful: A solution for the problem (1) can be absent. Physical sense of this effect is that a dielectric can perceive a limited amount of the electric fields that is interpreted as a loss of electrostatic equilibrium between externals source of electric field and dielectric. This effect we call as electrical puncture of dielectric.
In the work [3] it's shown, that analysis of electrostatic equilibrium in a dielectrics is reduced in order to research parameter 0 * > t , such, that for ( ) is limited on V and it's proved, that for * t an estimation above Solution of the given problem is reduced in [3] to substit (4) by the solution of next problem: and more precisely to research of properties of function R(u), having next view: It is possible to show by a simple example, that without boundary conditions, i.e. without fixing actual value of first or last variable, an objective function (6) achieves at minimum on a linear variety. In other words, an objective function has an endless number of points of minimum. For example, function of three variables ( ) for all variables fulfilling a condition: Through the variational-difference method based on finite-element piecewise linear approximation of the required solution, an initial continual problem (5) is reduced to the next finite-dimensional problem of minimization of uneven function of kind: The function (7) is a convex function, but it is nonsmooth because it's a sum of the modules, that does not allow to apply the standard methods of minimization, for example, gradient or Newton's methods. Even more, the simplest coordinate descent minimization requires a continuously differentiable (smoothing) objective function for the convergence [8].
For application of standard methods of minimization it is necessary to smooth out this function, i.e. to carry out it's regularization. is the regularization near x=0. We suggest to use the third variant -regularization near x=0, since it's more natural, saving basic properties of modulus function in the neighbourhood of singular point.
As a result, a finite-dimensional problem takes the following form: where ε is a small positive parameter ( 1 << < 0 ε ). In this case the object function is ill-conditioned.
Ill-conditioned functions are characterized by that slight changes of basic data can result in the massive changes of solutions.
We remind that the condition number of a matrix A is In practically important problems the condition number can evaluate too large as (10 6 -10 9 ). Function is named ravine, if its hessian can be ill-conditioned matrix.
The original objective function is attributed to the class of functions of ravine type, for minimization of which a number of the special methods has been worked out. We have chosen the heavy-ball method, since this method is master of the certain set of properties allowing effectively resolve problems of minimization of ill-conditioned functions [9].
Heavy-ball two-step method is lookup method of global minimum and its modification of gradient method.
Finding a solution of the problem with the help of gradient methods consists of some steps: beginning from some starting point n x are realized consequent steps to some other points unless acceptable solution of problem will be found.
Gradient method for solution of problem of absolute minimization where f: R m → R, it is possible to interpret in terms of usual differential equations in such a manner. We will consider next differential equation (here the above point x is the derivative on the independent variable t, and f ′(x) as usual means the gradient of mapping: R m → R; it is assumed that p > 0). The simplest finite-difference analogue of equation (9) has the view of obvious chart of Euler gradient method for problem (7) is written as Consider now instead of equation (9) the following equation describing motion of the ball of mass m in the potential field f ′ in presence of friction force. The loss of energy on a friction will force a ball to get down pithily a minimum of potential of f, and forces of inertia will not allow to oscillate how it is represented on a chart. It allows to hope that change of equation (9) by introduction an inertial member will improve the convergence of gradient method (10). The finite-difference analogue of equation, describing motion of a ball: After simple transformations and obvious labelling we get a next iteration scheme: that describes the heavy-ball method to solve the problem of absolute optimization of ravine functions.

Numerical Experiments
Numerical experiments were carried out in numerical computing environment and fourth-generation programming language Matlab. In the program was entered numerical vector of coefficients ψ , parameter of regularization ε, parameters α and β for calculations according to the heavy-ball method and precision of calculations δ.

Program Development
The program in Matlab was carried out in the form of M-files. The modules were implemented by corresponding M-functions.
The module of data control checked nonnegativeness of precision δ and parameter ε and was implemented in the body of M-function for calculations according to the heavy-ball method.
Module solving problem according to the heavy-ball method was implemented as a separate M-function. For that function parameters α, β, ε and precision δ were fixed by user through the Command Window.
The module for the calculation of gradient was implemented as a separate M-function called in the process of calculations in the module, solving the problem according on the heavy-ball method.

Testing of the Program
Before using the program, it was necessary to carry on testing with the aim of verification of all functions that the program must execute and verification of a response of the program in different situations.
It was necessary to conduct next tests to define influence of different factors, such as: -precision δ; -parameter of regularization ε; -parameters α and β for the heavy-ball method. For the verification of the program the method of testing described above was used on the example of the problem a decision of that is easily determined analytically, or in the Matlab.
With the aim of the comparison of the efficiency of different algorithms special test functions having essential singularities on extremum lookup are developed. One of well known is a Rosenbrock function, that was used for testing of the complex program.
For testing the program vector of coefficients with dimensionality n random number generator was used.
For estimation of influence of the precision on the quantity of iterations parameter ε and precision of calculations δ were changed with constant dimensionality of the vector of coefficients and parameters α and β.
For estimation of the sensitivity of heavy-ball method to the parameters α and β with the constant dimensionality of the vector of coefficients, precision and parameter of regularization parameters α and β we changed itself in given diapason and quantity of iterations was measured.
For estimation of influence of the parameter of regularization ε on the quantity of iteration with the constant dimensionality of the vector of coefficients, parameters α and β and precision δ was changed parameter of regularization itself in given diapason and quantity of iterations were measured. We see distinctly, that at ε=0.1 the number of iterations rises sharply and a process loops at α>0.15, while at ε<0.1 the number of iterations rises sharply already at α>0.12.

Results of Numerical Experiments with Considering of Boundary Conditions
The analysis of problem showed that objective function without boundary conditions has an endless great number of points of minimum, located in a ravine area that results in the formal cycling of algorithm. As a result, for providing of unicity of decision the boundary condition of х(1)=2 was used. We will notice that an initial variation problem in natural way contains boundary conditions on the required function.

Research of Sensibility of Heavy-Ball Method to
Parameters ( β α , ) General view of chart of dependence of number of iterations from parameters α and β at precision δ=0.1 and ε=0.01. At different precision δ the peak of number of iterations is observed at different values α, that evidently from the brought 4 th charts of dependence of number of iterations from the parameter α at the fixed parameters ε=0.01 and β=0.01.
With the change of precision the characteristic peak of number of iterations evolves in the area of small values, in this process his size raises sharply. From a chart is evident, that at precision δ=0.1 the number of iterations is small, at the increase of precision to δ=0.01 it increases sharply and then it changes insignificantly. Also it is obvious, that at the change of parameter ε the number of iterations changes in the narrow range of values. Thus, it is possible to conclude that the parameter of regularization ε poorly influences on convergence of heavy-ball method.

Basic Conclusions Based on Numerical Results
As a result of the carried out numerical experiments it was discovered, that without boundary conditions the objective function has an endless great number of the points of minimum, located in a ravine area, that result in formal cycling of algorithm. As consequence, for providing of unicity of decision it is necessary to use boundary conditions.
During numerical experiments with boundary conditions it was discovered, that: -At different precision δ the peak of number of iterations is observed at different values α. With the change of precision δ the characteristic peak of number of iterations evolves in the area of small parameters, there peak increases sharply and then it changes insignificantly.
-At precision δ=0.1 the number of iterations is small. At the increase of precision to δ=0.01 the number of iterations increases sharply and then changes insignificantly.
-Number of iterations at the change of parameter ε changes in the narrow range of values. As a result, it is possible to conclude that the parameter of regularization ε poorly influences on convergence of heavy-ball method.
-With decrease of parameter of regularization the number of iterations of heavy-ball method rises nonlinearly.
-The number of iterations at different values of parameter ε changes insignificantly, as well as at different values of precision δ.