**Abstract: ** Waveguide trapped modes (or bound states of quantum billiards, or
guided waves for diffraction gratings) are examples of
non-uniqueness for the Laplacian or other elliptic operators. Both
theoretical and numerical study of them has a long history. We
suggest a new approach to detect trapped modes numerically. It is
based on an existence criterion of trapped modes connected with
the spectrum of a so-called augmented scattering matrix (ASM).
This unitary matrix takes into account not only oscillating modes
but also finitely many of those which grow (attenuate) in
amplitude at infinity. Dealing with exponentially growing
solutions causes difficulties for numerical analysis. We introduce
a method which reduces computation of ASM to minimization of a
quadratic functional. To get the functional one has to solve an
auxiliary boundary value problems in the domain truncated at a
finite distance R; the minimizer of the functional exponentially
tends to the actual ASM as R goes to infinity. The numerical
scheme is not sensitive to the geometry of the problem and, in
particular, can be applied to a computation of the scattering as
well. In the talk we present a wide range of new examples
concerning trapped modes and guided grating waves. This is common
work with B.A. Plamenevskii and P. Neittaanmaki.