Dispersion fitting tool

See on github, run on colab, or just follow along with the output below.

Here we show how to fit optical measurement data and use the results to create dispersion material models for Tidy3d.

Tidy3D’s dispersion fitting tool peforms an optimization to find a medium defined as a DispersionModel that minimizes the RMS error between the model results and the data. This can then be directly used as a material in simulations.

[10]:
# get the most recent version of tidy3d
!pip install -q --upgrade tidy3d

# make sure notebook plots inline
%matplotlib inline

# first import packages
import matplotlib.pylab as plt
import numpy as np

import tidy3d as td

Load Data

The fitting tool accepts numpy arrays for the data points, but if we have a data file instead, we can load that with the load_nk_file utility function.

Our data file has columns for wavelength (um), real part of refractive index (n), and imaginary part of refractive index (k). k data is optional.

Note: load_nk_file uses np.loadtxt under the hood, so additional keyword arguments for parsing the file follow the same format as np.loadtxt.

[11]:
from tidy3d.fit import load_nk_file

fname = 'https://raw.githubusercontent.com/flexcompute/tidy3d-notebooks/main/data/nk_data.csv'

# note that additional keyword arguments to load_nk_file get passed to np.loadtxt
wavelengths, n_data, k_data = load_nk_file(fname, skiprows=1, delimiter=',')

# lets plot the data
plt.scatter(wavelengths, n_data, label='n', color='firebrick')
plt.scatter(wavelengths, k_data, label='k', color='chocolate')
plt.xlabel('wavelength ($\mu m$)')
plt.ylabel('value')
plt.title('refractive index data')
plt.legend()
plt.show()
../_images/examples_Fitting_3_0.png

Creating Dispersion Fitter

Now that we have our data as numpy arrays, we can perform the fit.

For this we load the data into the DispersionFit class, which provides various methods for fitting, visualizing, and exporting the models.

[12]:
from tidy3d.fit import DispersionFit

fitter = DispersionFit(wavelengths, n_data, k_data)

Fitting the data

The fitting tool uses the nlopt package to fit a dispersion model to the data by minimizing the root mean squared (RMS) error between the model n,k prediciton and the data at the given wavelengths.

There are various fitting parameters that can be set, but the most important is the number of “poles” in the model.

For each pole, there are 4 degrees of freedom in the model. Adding more poles can produce a closer fit, but each additional pole added will make the fit harder to obtain and will slow down the FDTD. Therefore, it is best to try the fit with few numbers of poles and increase until the results look good.

Here, we will first try fitting the data with 1 pole and specify the RMS value that we are happy with (tolerance_rms).

Note that the fitting tool performs global optimizations with random starting coefficients, and will repeat the optimization num_tries times, returning either the best result or the first result to satisfy the tolerance specified.

[13]:
poles, rms_error = fitter.fit(
                        num_poles=1,
                        tolerance_rms=1e-2,
                        num_tries=50,
                        globalopt=True,
                        bound=np.inf)
best RMS error so far: 9.95e-02: 100%|██████████| 50/50 [00:03<00:00, 12.70it/s]
        warning: did not find fit with RMS error under tolerance_rms of 1.00e-02
        returning best fit with RMS error 9.95e-02

The RMS error stalled at a value that was far above our tolerance, so we might want to try more fits.

Let’s first visualize how well the best single pole fit captured our model using the .plot() method

[14]:
fitter.plot()
plt.show()
../_images/examples_Fitting_9_0.png

As we can see, there is room for improvement at short wavelengths. Let’s now try a two pole fit.

[15]:
poles, rms_error = fitter.fit(
                        num_poles=2,
                        tolerance_rms=2e-2,
                        num_tries=100,
                        globalopt=True,
                        bound=np.inf)
best RMS error so far: 1.77e-02:  94%|█████████▍| 94/100 [00:19<00:01,  4.95it/s]
        found optimal fit with RMS error = 1.77e-02, returning

[16]:
fitter.plot()
plt.show()
../_images/examples_Fitting_12_0.png

This fit looks great and should be sufficient for our simulation.

Exporting to Medium

With the fit performed, we now need to construct a td.Medium to use in our simulation.

Method 1: direct export as Medium

The latest fit result can be directly exported as a tidy3d Medium with the .as_medium method.

This is useful if your fit is being performed in the same notebook as your simulation and you may want to try multiple fits.

[17]:
# direct export last fit result as medium
medium_si = fitter.as_medium(name='my_medium')

# construct a structure using this medium and use in simulation
my_structure = td.Box(center=[0,0,0], size=[1,1,1], material=medium_si)

Method 2: print medium definition string

In many cases, one may want to perform the fit once and then hardcode the result in their tidy3d script.

For a quick and easy way to do this, the fitting tool’s print_medium method prints out a string that can be copied and pasted into your script to hardcode the results.

[18]:
fitter.print_medium(name='my_medium')


COPY AND PASTE BELOW TO CREATE MEDIUM IN TIDY3D SCRIPT
========================
td.DispersionModel(
    poles=[
        ((-57033263687992.3+6326962248671373j), (99459487300.48624-1.837644146833881e+16j)),
        ((-1313217439474388.5+1501915617242655.8j), (920594391068434.8-5915997023941032j)),
    ],
    name='my_medium'
)
========================

Method 3: save and load file containing poles

Finally, one can save the poles from a fit in a file and load them to use in medium. Here is an example.

[19]:
# save poles to pole_data.txt
fname_poles = 'data/pole_data.txt'
fitter.save_poles(fname=fname_poles)

# load a medium directly from file
from tidy3d.fit import load_poles
medium_loaded = load_poles(fname=fname_poles)

# works just the same
my_structure = td.Box(center=[0,0,0], size=[1,1,1], material=medium_loaded)

Tricks and Tips / Troubleshooting

Performing dispersion model fits is more of an art than a science and some trial and error may be required to get good fits. A good general strategy is to:

  • Start with few poles and increase unitl RMS error gets to the desired level.

  • Large num_tries values can sometimes find good fits if the RMS seems stalled. it can be a good idea to set a large number of tries and let it run for a while on an especially difficult data model.

  • Tailor the parameters to your data. Long wavelengths and large n,k values can affect the RMS error that is considered a ‘good’ fit. So it is a good idea to tweak the tolerance to match your data. Once size does not fit all.

Finally, there are some things to be aware of when troubleshooting the dispersion models in your actual simulation:

  • It is common to find divergence in FDTD simulations due to dispersive materials. Besides trying “absorber” PML types and reducing runtime, a good solution can be to try other fits. Therefore, if a simulation is diverging with a single fit result, it is often worth fitting a few more times and trying those results.

  • If you are unable to find a good fit to your data, it might be worth considering whether you care about certain features in the data. For example, if the simulation is narrowband, you might want to truncate your data to not include wavelengths far ourside your measurement wavelength to simplify the dispersive model.