Posts |

Supercharging Your Job Hunt with ChatGPT and Espanso

Thu, 18 May 2023 19:39:48 -0400

Job Searching with ChatGPT

The meteoric rise of ChatGPT since its launch has made it a hot topic everywhere. An ever-increasing array of courses and articles dedicated to refining the art of prompting with language learning models (LLMs) emerges every second. Here is a great of effective prompting snippets that I always come back to.

Recently, I found myself back in the job market due to a corporate restructure. While thinking about my next move, it struck me - why not harness the prowess of ChatGPT to streamline my job search? A quick google search (I haven’t jumped ship to Bing 😏) led me to this great video by Jeff Su ( , ).

His insightful videos underscores some facets of job-hunting where ChatGPT excels:

Customizing resumes and cover letters according to the job description
Optimizing resume bullet points using successful templates
Crafting incisive questions for the company/interviewers
Drafting personalized LinkedIn messages to secure coffee chats
Interview preparation (compiling prepared questions and answers)
Composing impactful ‘Thank You’ emails
Negotiating job offers

Given that my job hunt has been progressing at a slower pace than I’d like, this post will focus on the utilization of ChatGPT in tailoring cover letters and resumes. Moreover, I will be discussing how Espanso can expedite these complex prompting steps.

Use Espanso to speed things up

The intricate prompting steps mainly comprise:

Assigning an identity to ChatGPT - think of a resume writing maestro with two decades of experience, or a seasoned tech recruiter, etc.
Providing comprehensive context, summarizing the previous step’s output, and incorporating it into the next prompt.
Constraining the output format, for instance, into a predefined table.
Specifically instructing ChatGPT to refrain from fabricating information.

It wasn’t long before I realized that repeatedly typing the same prompts was anything but fun. That’s when , a tool I’d previously utilized, sprung to mind. Espanso is an open-source text expander. Think of it as an auto-complete feature for modern IDEs that you can use anywhere you can type.

Getting Started with Espanso

Download Espanso binary , follow the setup instructions and provide necessary permisions.
In terminal, espanso status will show if espanso is running in the background, or check the icon in the status bar.
A simple example and some pre-built keywords :espanso will automatically change into Hi there!
To modify Espanso configuration and add more keywords and match rules, enter espanso path to locate the configuration folder.
The configuration folder, something like /Users/<username>/Library/Application\ Support/espanso (Mac for example), contains two subfolders, config/default.yml and match/base.yml

The files contained in the match directory define WHAT Espanso should do. In other words, this is where you should specify all the custom snippets and actions (aka Matches).

The files contained in the config directory define HOW Espanso should perform its expansions. In other words, this is were you should specify all Espanso’s parameters and options.

Whenever you alter the configuration, remember to use espanso restart to ensure the changes are active.
Press Alt + Space (or Opt + Space for Mac) to pull up the search bar for available keywords.

You can find more details .

Sample triggers

Here are some of my Espanso match rules specifically for ChatGPT. I’ve bracketed the job description and resume since they frequently need adjusting to fit specific job requirements. The “|” allows multiple lines to replace when setting the matching rules.

matches:
 # Simple text replacement
 - trigger: ":jdsum"
 replace: |
 Based on the job description, what are the biggest challenge for the position will face day-to-day.
 Please see quoted part between brackets for the job description.
 {job description}
 - trigger: ":jdskills"
 replace: "Based on the job description, List out the necessary skills by categories into a table."
 - trigger: ":covergrab"
 replace: |
 "Based on the resume posted below. {resume}
 You are currently working as a Data Scientist at a startup with 5 years of inter-disciplinary experience and you would love to apply for the position based on the job description.
 Write an attention-grabbing hook for you cover letter that highlights your experience and qualifications in a way that shows you empathize and can successfully take on the challenges of the data scientist role.

 Consider incorporating specific examples of how you've tackled these challenges in your past works and explore creative ways to express your enthusiasm for the opportunity. Keep your hook within 100 words."
 - trigger: ":coverall"
 replace: |
 "Incorporate the attention grabbing hook above, and compose a cover letter under 300 words.
 Please do not make up information, and focus on the enthusiasm of the opportunity."

Hope these help! Stay commited, stay positive!

Wildfire Smoke is Choking Seattle: visualizing using Altair

Wed, 22 Aug 2018 11:07:23 +0000

Last week was tough for everyone in Seattle because of the bad air quality. Sadly, it was nothing compared to what we are going through this week. The Air Quality Index (AQI) reached well above 150 since Monday, which is roughly equal to “ .” The hazardous smoke is attributed to the surrounding massive wildfires, and the onshore winds to the rescue are expected to arrive first in the Wednesday evening.

Here is the experimental smoke forecast from NWS’ tweet on Tuesday:

Another round of #smoke from the wildfires in north central Washington and British Columbia is expected to impact the area this afternoon. This will keep smoke thick, at least through Wednesday. Smoke conditions at: https://t.co/cgLa9sAQBQ #WAwx pic.twitter.com/cgJHU5SS7u
— NWS Seattle (@NWSSeattle) August 21, 2018

News articles seem to be obsessed with the AQI number. When I tried plotting out the AQI levels in Seattle in the past week, I realized that the AQI is unitless, but the data I found has a typical aerosol measurement unit $\mu g/m^{3}$. So I dived deeper to find out more details about AQI, which I’d like to explain below.

According to the EPA website, the AQI focuses on health effects, not strictly bounded to particle concentration. An AQI value of 100 corresponds to the national air quality standard for that pollutant, and EPA currently published AQI for four pollutants, ground level ozone, particle pollution, carbon monoxide, and sulfur dioxide. The number we saw in the news articles is for the particle pollution, specifically the PM2.5. Because the AQI 100 depends on the national standard, the AQI scale varies between countries and the regulatory agencies. Here is the current scale published by EPA:

https://airnow.gov/index.cfm?action=aqibasics.aqi

However, the conversion between AQI and particle concentration is not linear. It is also not clear to me, what the averaging period of the reported AQI number is. After some googling, I found a published by EPA, but the conversion is based on the 24-hr average of particle concentration.

The station-based PM2.5 data is harder to find than I expected. Kudos to the Puget Sound Clean Air Agency for providing a fantastic online and the downloadable data. Below I plotted the PM2.5 concentration in the past 10 days using Altair, a declarative statistical visualization library for Python, based on Vega and Vega-Lite.

Last Wednesday, the 24hrs-averaged PM2.5 concentration reached 95.4 $\mu g/m^{3}$, which is at the Unhealthy level. Yesterday the PM2.5 peaked at 154.9 $\mu g/m^{3}$, which is classified Very Unhealthy. Fortunately, the index seems to be declining in the past 24 hours.

Walk-through of the plot:

Clean up the dataframe: Altair takes in dataframes as data source. I changed the dtypes and the name of my two columns. One limitation is that Altair doesn’t recognize the index. So anything you want to plot need to be in columns (use reset_index)
Define the AQI levels as a dataframe source, set the legend and corresponding colors and plot the rectangles using: alt.Chart(source).mark_rect()
The line plot of PM2.5 with vertical line and circle while mouse hovering is composed of five elements (or layers): selectors, rule, line, point and text. This is adapted from in the Altair gallery.
Combine the line plot and the rectengle, simply (rect+layer).configure_axis(grid=False)

To my knowledge, the interactive plots you’ve seen on the Internet mostly involve javascript. Altair is no exception. It’s a wrapper that translates your python commands to -readable json objects. I really like the idea of “visualization grammar” that divides visualization into several building blocks. Vega/Vega-lite can take json objects and create rich interactive visualizations. I’ve been following Altair for quite a while, but only recently I decided to give it a spin. It’s my first plot using Altair. I’ll share more thoughts about Altair in future posts.

Wildfires and climate change

Thu, 16 Aug 2018 19:01:23 +0000

I moved to Seattle last summer. And I’ve already experienced three times of smoke this bad. Ironically, I had this impression that Seattle is famous for its good air quality. As many articles pointing out, the wildfires in the surrounding areas are to be blamed. They also suggest that the fires seem to spread wider and last longer than ever before due to the abnormally hot and dry condition this summer. Is it somewhat related to global warming?

To begin with, I found the active wildfire dataset from the . Two resolutions are available MODIS 1km and VIIRS 375m. You can choose between different regions and time periods. And the MODIS reflectance image is downloaded from the interface.

I used geopandas to read the shapefiles, and cartopy to plot the wildfires on the continental U.S.

import cartopy.io.shapereader as shpreader
from cartopy.feature import ShapelyFeature
import cartopy.crs as ccrs

# Change the reference system of Geopandas dataframe to the one Cartopy knows.
crs_epsg = ccrs.epsg('3857')
fp = "modis_fire_last7_2018_234_conus_shapefile/modis_fire_last7_2018_234_conus.shp"
data = gpd.read_file(fp)
df_epsg = data.to_crs(epsg='3857')

#ax = plt.axes([0, 0, 1, 1],projection=ccrs.LambertConformal())
fig, ax1 = plt.subplots(1, 1, subplot_kw={'projection': crs_epsg},figsize=(10, 5))
ax1.set_extent([-125, -66.5, 20, 50], ccrs.Geodetic())

shapename = 'admin_1_states_provinces_lakes_shp'
states_shp = shpreader.natural_earth(resolution='110m',category='cultural',
 name=shapename)
ax1.background_patch.set_visible(False)
ax1.outline_patch.set_visible(False) # Remove the outline, very useful
#ax.coastlines()

for state in shpreader.Reader(states_shp).geometries():
 # pick a default color for the land with a black outline,
 # this will change if the storm intersects with our track
 facecolor = 'LightGray'#[0.9375, 0.9375, 0.859375]
 edgecolor = 'black'

 #if state.intersects(track):
 # facecolor = 'red'
 #elif state.intersects(track_buffer):
 # facecolor = '#FF7E00'
 ax1.add_geometries([state], ccrs.PlateCarree(),
 facecolor=facecolor, edgecolor=edgecolor,alpha=0.6)
df_epsg.plot(ax=ax1, color='red',markersize=7,zorder=51)
plt.title('Wildfire 8/15-8/22 from MODIS satellite')
#plt.savefig('MODIS_map_last_7days.png',dpi=200)
plt.show()

I downloaded the wildfire occurrence data from the . Here I focus only on the data collected by US forest service. The metadata can be found . Again, I used geopandas to read the shapefile, and converted interested columns to pandas dataframe. I then changed some columns to the correct dtypes and selected only the major wildfires occurred in the western U.S. The major wildfire is defined as those fires burned over 1000 square acres when controlled.

import pandas as pd

ff = "wf_usfs_1980_2016/wf_usfs_1980_2016.shp"
df = gpd.read_file(ff)

dff=pd.DataFrame(df[['CAUSE','SPECCAUSE','SIZECLASSN','YEAR_','STARTDATED',
 'CONTRDATED','OUTDATED','STATE','TOTALACRES','DLONGITUDE']])

# Clean data
dff['STARTDATED']=pd.to_datetime(dff['STARTDATED'])
dff['CONTRDATED']=pd.to_datetime(dff['CONTRDATED'])
dff['OUTDATED']=pd.to_datetime(dff['OUTDATED'])
dff['LENGTH']=(dff['CONTRDATED']-dff['STARTDATED'])

dff['YEAR']=pd.to_numeric(dff['YEAR_'])
dff=dff.drop('YEAR_',axis=1)

WEST_US_major_fire=dff[(dff['SIZECLASSN']>5) & (dff['DLONGITUDE']<-103)]

Three questions I’d like to answer:

Has the occurrence of major wildfires increased in the last 30 years?
Has the wildfire season got longer in the same period?
Do we have more severe wildfires in recent years?

event_count=WEST_US_major_fire.groupby('YEAR')['STARTDATED'].count()
total_acre=WEST_US_major_fire.groupby('YEAR').sum()['TOTALACRES']
#year=pd.date_range(start='1980', end='2017', freq='A')
season=pd.Series(season_window,index=total_acre.index)
season[season>365]=365
annual_df=pd.DataFrame({'event_count': event_count, 'total_acre': total_acre,
 'season_length':season})

I plotted the wildfire occurence, burned area and season length against the temperature and precipitation reanalysis data I downloaded from using Altair.

There are clear increasing trends in the average and maximum surface air temperature, as well as a decreasing trend in precipitation over the western U.S. since 1980. The occurrence, burned area, and season length of wildfires all appear to increase during the same period. The burned area and occurrence seem to have strong interannual variability. The average wildfire season length after 2000 is about 70 days longer than that in 1980-2000.

This is a very crude exploration of data. I am fully aware of the danger of mistaking correlation as causation. There are more things to be done, more variables to be considered. For example, after reading through several journal articles, I learned that soil moisture also plays an important role. Unfortunately, the soil moisture is scarcely measured and poorly modeled. I would also like to expand my research by including more wildfire data from other sources.

Parsing a .xlsx file generated by Labchart

Thu, 16 Aug 2018 15:26:08 +0000

My wife often complains about manually repeating a lot of steps for her research. With a little python skill, I would love to help. So I asked her what she’s been working on recently. One of her experiments aims to measure the change of pressure and volume of a mouse’s heart, the pressure-volume loop of heart contraction/relaxation.

The data are collected through LabChart Pro, and there are options to output to different formats, including .txt, .xlsx, .mat. Throughout her career, her colleagues and she mostly use Excel to process and to plot the data. However, she doesn’t know any fancy tricks of Excel, and often hanlded the data manually. The more mice used for the experiments, the more times she needs to repeat. And without remembering all the processing steps, she can’t be sure that all data-processing is done consistently. It leads to another hot topic in all fields of science in recent years: the lack of reproducibility.

Step 1: Data inspection

I first looked into the .xlsx file she gave me. The original measurements are stored into four columns: time, pressure, volume and loop, together with many lines of header summarizing the experiment setup. I decided to skip the headers for now. One file measures one mouse for a specified period. For this specific case, there are twelve PV-loops, whose starting time-steps are marked in the loop column.

import pandas as pd
import matplotlib.pyplot as plt

# Load in data using pandas:
cols = ['Time','LV Pressue','LV Volume', 'Loop']
df= pd.read_excel('336-ivc2.xlsx',names=cols, usecols = "A:D",skiprows=140)

Step 2: Data cleaning

One thing I noticed, the loop column only marks the starting time-step of each loop. I want all rows from the same loop to have the same flag. I first filled the missing rows with the last available value, then extracted the loop number and converted the column to numeric.

df=df.fillna(method='ffill') # Foward Fill the missing value
temp=df['Loop'].apply(lambda x:x[12:14])
df['Loop']=pd.to_numeric(temp.str.replace(';', ''))

Step 3: Plotting/Groupby/Processing

Now I can easily select and visualize each loop’s PV changes.

df[df['Loop']==1]['LV Volume'].plot()

The goal is to visualize the mean of all 12 PV loops. To do that, I first created a new Step column to index the steps in each loop.

def restep(series):
 length=len(series)
 new_steps=[x for x in range(1,length+1)]
 return new_steps
for loop in range(1,13): # Must inspect the .xlsx first
 df.loc[df['Loop']==loop,'Step']=restep(df.loc[df['Loop']==loop,'Step'])

I created another column, PdV, defined as $\frac{P}{V-V_{0}}$. This is the variable we want.

df['PdV']=df['LV Pressure']/(df['LV Volume']-3.902683)

The twelve PdV loops look like this:

df.PdV.plot()

The Step column comes in handy to plot the mean of the twelve loops.

import seaborn as sns
sns.set_style("white")

# Groupby eman
gbmean = short_df.groupby('Step')['PdV'].mean()
ax=gbmean.plot()
plt.title('mean PdV of 12 loops')
plt.xlabel('Step')
plt.ylabel('PdV')
sns.despine()
plt.show()

Pandas is powerful, but I am pretty sure that some Excel gurus can do the same thing quickly. It’s fun to work on data that I have zero knowledge of. I’ll try to rewrite some of the processing steps into functions that my wife can easily apply to the new .xlsx from the same experiment.

Remove model drift using control run simulations

Thu, 05 Oct 2017 00:16:43 +0000

Recently, I’ve been trying to remove model drift from my high-resolution CCSM 20th century climate change simulation. The model drift is estimated using the two companion control runs, HRC08 and HRC09. All three runs were spun up from a similar initial condition, with CO₂ held fixed in year 2000 level. Following Gupta et al., [2013]¹, we tried to identify the model drift by fitting a cubic polynomial to the full record of control runs (nearly 70 years). The drift is then removed from the 20th-century climate change simulation. One thing is worth pointing out, this climate change run did not start from a pre-industrial level of CO₂, but a year 2000 level forcing. Some variables, including the global mean sea surface temperature, may have lingering effect of higher CO₂ level that takes few decades to reach the new equilibrium.

Below, I’d like to document my experience of cubic fitting the surface temperature (TS) locally at each grid point. Removing these drifts from the TS in the climate change run yields the de-drifted TS dataset. This post highlights several packages I’ve been using most frequently, including numpy, xarray and cartopy.

f1 = './data/TS.ens.all.nc'
ds1 = xarray.open_dataset(f1, decode_times=True, chunks={'lat':200,'lon':200})
TS_annual=ds1.TS.resample('12MS',dim='time')

These lines load in a NetCDF file with 69 years of monthly mean TS (time: 69, lat: 384, lon: 576) in (200x200) chunks. The chunk is the built-in support of in . Xarray is a very powerful package. It attempts to extend the labeled data power of Pandas to the multi-dimensional datasets that we often work on in geophysical sciences.

import matplotlib.dates as dates
from numpy.polynomial.polynomial import polyval,polyfit
import xarray
def driftfit(arr):
 stacked = arr.stack(loc=('lat', 'lon'))
 timenum=dates.date2num(stacked.time.to_index().to_pydatetime())
 regressions = polyfit(timenum, stacked, 3)
 drift_stacked= polyval(timenum,regressions)
 foo = xarray.DataArray(drift_stacked.T, coords=[stacked['time'], stacked['loc']], dims=['time', 'loc'],name=arr.name)
 arr_drift=foo.unstack('loc')
 return arr_drift

This is a function to fit a 3rd order polynomial to the 69-year surface temperature of each grid cell. It stacks locations of the input array using arr.stack(loc=('lat', 'lon')), and using numpy.polynomial.polynomial.polyfit to find the cubic fit of the local temperature time series over 69 years. The coefficients are then converted back to the polynomials using polyval(timenum,regressions). Next step is to generate a DataArray from numpy.array so that I can unstack the array correctly and keep the metadata from the input DataArray.

Note

One interesting and confusing thing I noticed: not all polyfit are created equal. The numpy.polyfit outputs the polynomial coefficients from the highest power, while the numpy.polynomial.polynomial.polyfit does the other way round. And so are the input order of their corresponding polyval.

I then load in the climate change run and remove the cubic fit from it. Two functions lintrend and trendts are constructed similarly to driftfit, so I left out the details. The former output the coefficient of the first order polynomial (the slope), and the later output the trend time series.

ensfit=driftfit(TS_annual)

f2 = './data/TS.HRC07.all.nc'
ds2 = xarray.open_dataset(f2, decode_times=True, chunks={'lat':200,'lon':200})
TS_HRC07_annual=ds2.TS.resample('12MS',dim='time')
TS_HRC07_dedrift=TS_HRC07_annual-ensfit
trend_HRC07=lintrend(TS_HRC07_dedrift)
trendTS_HRC07=trendts(TS_HRC07_dedrift)

HRC07 TS timeseries at 40E, 45S, and the cubic fit from control runs.

After removing the drift from each location, I plot out the linear trend map of TS for the HRC07 run using cartopy.

plt.figure(figsize=(13,5))
ax = plt.subplot(111, projection=ccrs.PlateCarree()) #ccrs.Mollweide()
mm = ax.pcolormesh(ds1.lon,
 ds1.lat,
 trend_HRC07*365*10,
 vmin=-.5,
 vmax=.5,
 transform=ccrs.PlateCarree(),cmap=cmo.balance )
ax.set_global()
#ax.set_extent([-180, 180, -70, 70])
ax.coastlines();
cb=plt.colorbar(mm,ax=ax,fraction=0.046, pad=0.01)


#ax.set_xticks([0, 60, 120, 180, 240, 300, 360], crs=ccrs.PlateCarree())

ax.set_xticks([-180,-120,-60,0, 60, 120, 180], crs=ccrs.PlateCarree())
ax.set_yticks([-90, -60, -30, 0, 30, 60, 90], crs=ccrs.PlateCarree())
lon_formatter = LongitudeFormatter()
lat_formatter = LatitudeFormatter()
ax.xaxis.set_major_formatter(lon_formatter)
ax.yaxis.set_major_formatter(lat_formatter)


ax.set_title('dedrifted HRC07 Temperature linear trend ')
cb.set_label('degC/decade')

plt.show()

HRC07 TS linear trends from 1953 to 2008 with model drift removed

The cooling trends in the Agulhas System seem to contradict to what we anticipated and hence need a closer look. Note that this kind of trend and drift estimates highly depends on the chosen endpoints and available record lengths. In Gupta et al. [2013]¹, drifts are estimated from multiple control runs of hundreds of years. For our high-resolution model, having multiple century-long control runs is a luxury. Maybe this technique is not applicable given the short record of our runs. Besides, estimating the drift locally may need to be justified.

Gupta, A. S., N. C. Jourdain, J. N. Brown, and D. Monselesan, 2013: Climate Drift in the CMIP5 Models*. J. Climate, 26, 8597–8615, doi:10.1175/JCLI-D-12-00521.1. ↩︎ ↩︎

Updated Hugo Academic theme to sync the upstream repo

Mon, 27 Feb 2017 00:28:36 +0000

I just updated the Hugo-Academic theme from the upstream repo. The structure has been greatly changed. Now each section on the index.html is a widget. Changing Research Interests and Education is no longer inside the config.toml, but the about.md as metadata. I also added a Talk widget, and the Project widget now showcase my research in cards.

Below I would like to try out some new features:

Emojis!!!

Now the Hugo-Academic theme supports Emojis natively. Check for a cheat-sheet of available Emojis.

😃 😪 🎾 👍 🀄 Tomorrow is Vicky’s defense! 📅

Blockquotes

This is not part of the theme. I updated the custom css file to achieve this effect.

We live in a society exquisitely dependent on science and technology, in which hardly anyone knows anything about science and technology.

Carl Sagan

Syntax highlighting using highlight.js

Python:

from pandas import Series, DataFrame, Panel
import gsw
import scipy.io as sio
fulllist=sorted(glob.glob('/raid60/kirtman/HRC07/ocn/mo_nc4/*.nc'))

months=pd.period_range('1951-01', '2002-12', freq='M')
OHC300_HRC07=Series(OHC300,index=months)
OHC700_HRC07=Series(OHC700,index=months)
OHC2000_HRC07=Series(OHC2000,index=months)

Matlab:

MMend=str2num(end_date(6:7));
DDend=str2num(end_date(9:10));

for yrid=find(yyy==YYs):find(yyy==YYe)
 if (yrid==find(yyy==YYs))
 mstart=MMs;
 else
 mstart=1;
 end
 if (yrid==find(yyy==YYe))
 mend=MMend;
 else

For available languages and styles, check

Twitter Tweet

Running wild with @BearGrylls pic.twitter.com/RG3pRquOuu
— Roger Federer (@rogerfederer) February 19, 2017

Youtube video

Alerts

Note

It’s about time to sleep!

Warning

Hope this will help you.

Some more options in the Hugo-Academic . That’s all for now. I guess the only thing I missed about Pelican was its plugin to embed ipython notebook cells to the post. Haven’t found a hugo-shortcode for that purpose yet. Or maybe a shortcode to embed an Instagram post?

Wow, there is a built-in shortcode for Instagram!

View this post on Instagram

Using git-latexdiff to track changes in tex-file graphically

Sat, 25 Feb 2017 20:13:52 +0000

It’s been a while since I started using git. The version control system helps to track changes in computer files and coordinating work on those files among multiple people. Composing a manuscript is nothing unlike developing a program. If multiple authors work on the same manuscript, git can be very helpful to avoid conflicts. If you are the only author, git can help you to document what has been changed since the last commit or put your advisor’s comment in another branch.

I’m a big fan of $\Latex$. It’s powerful and elegant. With a given template, one should only focus on the content. However, when it comes to tracking changes, $\Latex$ is a lightyear behind Word. Fortunately, I recently bumped into a program latexdiff, which visually tracks changes between $\Latex$ generated pdfs, much like how Word does. Before, I often manually color the lines I changed when sending to my advisor.

git-latexdiff is a wrapper around git and latexdiff. Instead of manually generating dummy tex files of different names (and also the auxiliary files), git-latexdiff can graphically show the differences between various versions of a LaTeX-generated pdf. Below is an example of how I incorporate these tools to my workflow.

Git basics

You can turn any folders into git-tracking folders by typing the command git init . in the targeting folder.
git add .: add all files in the folder to staging
git commit: commit changes to head (the changes are still local)
These committed changes can then be pushed to a remote repository hosted on GitHub or gitlab.
git checkout -b branchname to create and switch to new branch (git checkout branchname to switch after commit if the branch already exists)
git status to check current branch and status
After you are satisfied with the changes you made in a developing branch, merge back to master by git merge branchname, assuming the current branch is master.
Many GUI git can save the day if you are not used to command lines. For now, I am using GitKraken.

Track changes

latexdiff works like this:

latexdiff draft.tex revision.tex > diff.tex

To compare the difference between first two versions of .tex, a third dummy diff.tex is created, along with many auxiliary files. Check this for better control over latexdiff. Using git-latexdiff, one can track the differences between the same file on different branches or versions, like this:

git latexdiff --latexmk --append-context2cmd="abstract" --main Largescale_eddy_interannual_AL.tex master advisor

git-latexdiff also has many additional options. More detailed information can be found and the gitlab page of .

Fill in the missing data using Python pandas

Mon, 13 Feb 2017 00:42:30 +0000

One of the many advantages of Python is its abundant and often powerful Libraries. For my research, besides plotting maps, I often play with time series. When it comes to manipulating and plotting time series, no other tools can beat python pandas.

pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language.

At the core of Pandas are the data structures: Series, DataFrame and Panel. The ones I used the most are the first two. A Series is an array labeled with timestamps, and a DataFrame consists of many Series. In a real-world use case, I use pandas to generate a range of time-axis, which is then attached to my Agulhas leakage time-series. After doing that, the value at a specific timestep can be easily retrieved by calling Series['timestamp']. And to plot the whole time series is as simple as Series.plot().

For a DataFrame, to see the key statistics of a DataFrame with many columns, simply use DataFrame.describe(). A table with mean, standard deviation, counts, and percentiles will then pop up. To compare multiple time series visually, naively put DataFrame.plot().

Working with missing data

Recently, I am calculating the Atlantic Ocean Heat Content (OHC).

#headers=['date','OHC2000','OHC300','OHC700']
OHC_multilevels=DataFrame.from_csv('OHC_HRC07_1951-2002.csv') # If it's pandas generated, this is much easier.
OHC_multilevels.plot()

Atlantic OHC in multiple layers 1951-2002

Obviously, something fishy happened near 1952 and again in 1971. Several months have values close to zero, which is unlikely. Going back to the data, I confirmed that the temperature and salinity fields of those months are missing. To clean up the time series, I first assigned None to those months, and interpolate linearly using the neighboring months. Three time series in the same DataFrame are processed using following two lines.

OHC_multilevels[OHC_multilevels<100]=None
OHC_multilevels.interpolate().plot()

filled missing data with linear interpolation

This is just a glimpse of the awesomness of pandas. More details can be found in the .

Gibbs SeaWater Oceanographic Package for Python

Sun, 05 Feb 2017 00:42:30 +0000

Recently, I’m looking into the Meridional Heat Transport across the 35$^{\circ}$S in our high-resolution CCSM simulation following . To calculate the heat transport, the following formula is used:

\begin{equation} \int \rho C_{p}v\theta dx dz \end{equation}

The tricky part is whether one should use temporal and spatial-varying values of $C_{p}$ (seawater heat capacity) and $\rho$ (seawater density). Since density and heat capacity are not among the standard outputs, I went on looking for some standard libraries to calculate them using the fields I have: temperature, pressure, and salinity. Soon, I found the Python version of Gibbs SeaWater Oceanographic Package.

The Gibbs-SeaWater (GSW) Oceanographic Toolbox contains the TEOS-10 subroutines for evaluating the thermodynamic properties of pure water (using IAPWS-09) and seawater (using IAPWS-08 for the saline part). You can download the toolbox in differnt languages through this . Also can be found are the abundant . The can be easily installed through the standard pip channel, simply type pip install gsw.

So far, I used several functions:

gsw.p_from_z(depth,latitude): caclculate in situ pressure (in dbar) from depth (in meter)
gsw.CT_from_pt(Salinity,Potential_Temperature): convert potential temperature (CCSM output) to conservative emperature
gsw.t_from_CT(Salinity,Conservative_Temperature,Pressure): convert conservative temperature to in situ temperature
gsw.rho_t_exact(salinity,temperature,pressure): calculate seawater density from field variables
gsw.cp_t_exact(salinity,temperature,pressure): calculate the heat capacity of seawater from field variables

For any physical oceanographers, the temperature, salinity and pressure trio cannot be missed. Considering the compressibility, potential temperature and potential density are usually introduced next. But, I don’t recall I have ever heard of Conservative Temperature. It struck me as the first time I learned about potential temperature.

Potential Temperature ($\theta$) and Conservative Temperature ($\Theta$) are actually not that different. They are defined similarly involving the thought experiment that adiabatically (no heating) bringing the deep water to the sea surface (p=0). The temperature of the fluid parcel after the procedure is the Potential Temperature, and the parcel has the enthalpy defined as “Potential Enthalpy”. Dividing the “Potential Enthalpy” by the fixed “heat capacity” ($c_{p}^{0}=3991.867 [JKg^{-1}K^{-1}]$) yields the Conservative Temperature. $\Theta$ better represents the “heat content” of seawater and because $\Theta$ is almost a perfectly conservative variable, the meridional “heat” flux is very accurately given by the meridional flux of $\Theta$ (as opposed to the meridional flux of $\theta$).

ref:

Jupyter notebook gallery

Sun, 05 Feb 2017 00:42:30 +0000

Sometimes I got emails requesting codes to plot certain things. Those inquiries remind me that I have an older version of my website on Wordpress.com still up and running. That was an answer to the requirement that every student in RSMAS should have their own websites back in 2014. To fill in the blanks, I listed out some of my python works in Ipython notebooks.

Python is a thing that I picked up in the graduate school. I not only use Python to visualize model outputs or satellite data but as a shell-script alternative to control my workflow. Ipython stands for Interactive Python. It was an interactive shell, complementing the original python prompt. The Notebook web-interface was added to Ipython project in 2011, and it has evolved into an enormous spin-off project called Jupyter in 2014. Such notebooks connect to different kernels. There are many kernels compatible with Ipython/Jupyter, including Python (natürlich!), Julia, R, and Haskell.

I love using Ipython notebooks because I can run code, comment in Markdown and plot inline on the same page. Everything is then easily exported to shareable formats such as pdf, HTML, Latex, even presentation slides.

Ths description quoted from Jupyter.org:

The Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text. Uses include: data cleaning and transformation, numerical simulation, statistical modeling, machine learning and much more.

In 2013, I participated in a research cruise “Meteor 100/2”. On board, I was asked to provide daily Satellite plots as well as plots from RTOFS outputs. I used Python scripts to control the daily procedure to download data, to plot these data with several NCL scripts, then tar and compress them before moving them to DropBox shared-directory, so that the Chief-Scientist on board could receive the latest satellite images, regardless of the pathetic bandwidth of the satellite-based internet.

Satellite SST and onboard observed wind-vectors

Like I said, Ipython Notebooks are easily shareable, in particular through the nbviewer provided by Jupyter.org. Below are some links to my early works in the notebook format:

Remove model drift using control run simulations

Thu, 06 Oct 2016 00:42:30 +0000

The University will be closed on both Thursday and Friday in preparation for the approaching Hurricane Matthew. Students and faculties were asked to leave RSMAS by 5 pm on Wednesday since wind shutters would completely seal the MSC building. The threat seems imminent; people are freak out – Publix was like a war zone when I was ordered to pick up some bottled water on my way home.

Last night, Matthew with the maximum sustained wind at 140 mph, was a category 4 hurricane (>130mph). Last time a major hurricane (category 3 and above) made a landfall in the US was back in 2005 (Wilma at 120 mph). I slowly began to understand peoples’ reactions.

More like overreactions?###

Growing up in Taiwan and coming from atmospheric sciences background, I have experienced many typhoons. Just recently, I visited Taiwan from 9/14 to 9/27. During that short period, three major typhoons (Meranti, Malakas, and Megi) hit the little island. My return flight to the US nearly got canceled – my wife and I were sitting on the grounded but shaky aircraft for almost an hour. So honestly, I am surprised to see my colleagues’ hurricane preparation.

I mean, it should be like this. General public are aware of the threats and cooperate with the government to evacuate. Schools and companies grant enough time for students and employees to get prepared for the approaching storm. There is no such thing as overpreparation. It’s the right attitude facing any threats at such magnitude. Instead, people in Taiwan always blame everything on the Central Weather Bureau and local governments without being sufficiently prepared.

check out recent typhoons in Taiwan!

How to read the NWS hurricane warning?

The best way to track the latest hurricane forecast is to visit the . I’m no hurricane/typhoon specialist (like I always said, I dived into the ocean when I came to Miami). Thanks to my colleagues, I finally got a grip on reading the NWS hurricane warnings. They are very different from the ones issued by Taiwanese weather bureau, and I would like to share some of my findings below:

NWS hurricane warning

Concise punch line on the top (as for the 2 am warning): …MATTHEW POUNDING PORTIONS OF THE CENTRAL BAHAMAS… …EXPECTED TO INTENSIFY AS IT APPROACHES FLORIDA…
The essential hurricane statistics on the top-left including location, moving direction and speed, minimum pressure and maximum sustained wind.
For more detailed paragraphs, checked the Public Advisory, Forecast advisory and discussion, and their Spanish versions. Check their official guide on I personally enjoy reading the Forecast advisory.
For the path forecast, check the . Over 2/3 of chances that the center of the hurricane will stay in the white zone on the map. However, the area it may affect is much broader, considering its rain bands and hurricane force winds. Also there is still 1/3 of chances that the center of the hurricane will be outside that cone. More details can be found .

forecast path
For the currently wind affected area, check the . It shows the maximum possible extent of a given wind speed within particular quadrants around the tropical cyclone. By definition, Hurricane force winds exceeds 74mph and Tropical Storm force winds are no less than 39mph, so the later cover a much larger area.

force wind affected area
Now comes to the These plots indicate the cumulative possibility that a specific area will experience winds over a certain threshold. Users can change the cumulative periods (up to 120hrs), and the wind thresholds (tropical storm force, 50knots and hurricane force.)

force wind speed probabilities
Rainfall and storm surges are more difficult to predict. But NWS also provides such products.

Others

I was surprised to see that Matthew reached 140mph last night. However, I realized that NWS takes 1-min average maximum sustained wind to define hurricane intensity. In West Pacific, 10-min average is used instead. So I’m less impressed by Matthew now. But let’s check back in 24 hrs since it is predicted to regroup and intensify again before reaching Florida.
Interesting read on the differences between watches and warnings:

Watches vs Warnings

References:

Automated deployment of Hugo generated site to Github pages

Wed, 14 Sep 2016 20:54:16 +0000

In the last post, I listed the typical workflow of generating a static website with Hugo. Here I would like to share the issues I encountered and the tweaks I’ve made. The source codes for this site can be found .

Basic setting in my config.toml for the Academic Theme

Many beautifully designed themes are already . I decided to use the , because it’s specifically designed for academic uses, showcasing publications, personal profile, and projects. For more details, please see the in action.

In config.toml, one can set the universal variables used by Hugo to generate HTML pages. For example:

baseurl = "https://fischcheng.github.io/"
title = "Yu Cheng"
copyright = "&copy; 2016 Yu Cheng"
languageCode = "en-us"
theme = "hugo-academic"

Also some specific parameters used by the chosen theme:

[params.about]
 interests = ["Largescale ocean circulation", "climate variability and modeling"]
 str_interests = "Interests"
 str_education = "Education"

[[params.about.education]]
 course = "BSc in Atmospheric Sciences"
 institution = "National Taiwan University"
 year = 2010

In the academic theme, there are 7 sections on the homepage. The about and teaching sections are under content/home. The content of personal profile is from about.md, yet the entries in interests and education are configured in config.toml.

Three other types of contents are publication, post and project. Each has its unique page layouts, stored under corresponding folders under content. Add a line selected = true in the meta-data part in the publication entry to ensure that entry displayed in the selected publication section.

Next change following items to customize the navigation bar:

[[menu.main]]
 name = "Home"
 url = "#top"
 weight = 1

[[menu.main]]
 name = "Publications"
 url = "#publications"
 weight = 2

[[menu.main]]
 name = "Posts"
 url = "#posts"
 weight = 3

[[menu.main]]
 name = "Research"
 url = "#projects"
 weight = 4

[[menu.main]]
 name = "Works"
 url = "#works"
 weight = 5

[[menu.main]]
 name = "Contact"
 url = "#contact"
 weight = 6

Changing the name will tell Hugo to replace the navbar items and URL for the page that button linked to. I modified the default teaching.md page to works and renamed project section to research. Also, I enabled commenting by setting disqusShortname = "yucheng" in config.toml. And follow this snippet to add a new social network icon:

[[params.social]]
 icon = "envelope" # icon name for email
 icon_pack = "fa" # for font-awesome icons
 link = "mailto:username@xxx.xxx"

On top of the hugo-academic theme, I added an additional stylesheet under static/css. This changes the default color scheme from light blue to green. Besides, I changed the font of the navbar-brand, the title of my site by adding following lines:

.navbar-brand {
 text-transform: uppercase;
 font-weight: bold;
 font-size: 1.5em;
 color: #2b2b2b;
 font-family: 'Permanent Marker', cursive;
}

The contents within static/js and static/ccs will be included in the header.html after the default ccs stylesheets, so it’s totally possible to change color scheme without modifying anything in the themes/academic. However, to change fonts, I modified themes/academic/layouts/partials/header.html to load an additional font Permanent Marker from google fonts.

<link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:400,700|Merriweather|Roboto+Mono|Permanent+Marker">

Github pages, user site or project site

Two kinds of site are supported on GitHub pages, the user/organization site, and the project site. For the former, the site itself is a repository, everything under master branch will be displayed under this address https://<username>.github.io, which is customizable. And the later is a website to showcase an existed repo. Without changing the original branch, everything under gh-pages branch can be accessed under http://<username>.github.io/<repository>. Github even provides an Automatic Generator to help to create a project site. More details can be found .

My website is an user site. Under this repo fischcheng.github.io, the source branch archived all changes to the source codes, and master branch store the Hugo-generated HTML pages.

Automated deployment using Wercker

The default way to build a hugo site is simple, invoke hugo under the site root. A public folder will be created, containing every HTML pages and copied stylesheets and static files (banner images and profile icons or so). Push this folder to the master branch, and voila, the site is online.

What if a single push to the source branch (or whatever name of your development branch) can trigger all the rest steps? Travis-CI was my old choice for the last version of this site, until I came across this wonderful guide on Hugo’s official . However, things changed a lot since that article was last edited and below I would like to detail all the issues I ran into and how I solved them.

is a continuous integration (CI) tool that helps developers to build, test and deploy their applications based on . A Wercker account can be easily hooked to a given Github or Bitbucket account, and one can create new applications from a chosen repository. After setting up, a push to the repository will automatically trigger the application. One of the biggest advantages of Wercker over Travis-CI is the collection of easily-located and well-documented steps. In our case, the application consists of two steps build hugo and deploy to GitHub.

Wercker dashboard: to find existed steps, click the Registry tab; to create your own applications and steps, access the Create tab.

The first task is to create the config.yml ( ) in the developing branch. Please follow the original Hugo guide to setup the link between a Wercker application to the Github. Below, I will only list out the discrepancies I encountered.

Build

Following the official guide, I used this to trigger Hugo to build HTML pages. However, my first try didn’t successfully generate the pages correctly, because the arjen/hugo-build step couldn’t find the hugo-academic theme. For easier tracking and updating, I forked the hugo-academic repo, and I later realized that, for a repo embedded within another repo, the former is counted as a submodule. For the build step to work as desired, I had to install git and initialize submodule to ensure that the build step can locate the hugo-academic theme. Kudos to this which helped me to track this issue down.

box: debian

build:
 steps:
 - script:
 name: install git
 code: |
 apt-get update
 apt-get install git -y
 - script:
 name: initialize git submodules
 code: |
 git submodule update --init --recursive
 - arjen/hugo-build@1.11.0:
 version: "0.15"
 theme: "hugo-academic"
 flags: --buildDrafts=true

Deploy

There is no more such thing as “Add deploy target” in Wercker. The interface has changed to the new “workflows of pipelines.” This change is so new that even the documentation on Wercker website has not changed yet. Their customer service still referred me to the old supporting materials.

So I poked around and realized that, even if the second deploy step is included in the wercker.yml, one still needs to manually add a new “pipeline” under the “workflow” tab. Upon creating a new pipeline, the “YML pipeline name” must be in the predefined names in the wercker.yml.

Here I used this to deploy the built site to GitHub. Each pipeline starts from scratch. So for the deploy pipeline, the git package needs to be installed again. One also has to setup the environment variable $Git_Token, acquired from Github setting.

deploy:
 steps:
 - script:
 name: install git
 code: |
 apt-get update && apt-get install git -y
 - leipert/git-push@0.7.6:
 gh_oauth: $Git_Token
 gh_pages: true
 basedir: public
 branch: master

Click deploy pipeline to setup included environment variables. Get the Github access token from Github profile setting.

Switch from Pelican to Hugo

Wed, 14 Sep 2016 01:16:30 +0000

Being a big fan of all things Python, I built the last two incarnations of this site with Pelican. I even went to the length to set up a Travis CI to automatically build and deploy to GitHub pages. That was the beginning of the summer and only two new posts ever since.

Recently, my boss has brought up the topic of building a group website. Also, there is a joint-project website needs to be done. Most other groups in the school have their group website, mostly hosted by Weebly, Wordpress or Wix. Those website services certainly have pros. For one, a Weebly site can easily allow multiple contributors. But most customization and advanced features are usually not free.

So, I shared my experience of hosting a static site on Github pages and showed off my Pelican-powered website. By generating your own website, hosted by some popular cloud services (i.e. Amazon S3 and Github), one has full control to the contents and styles. All other responsive features such as site analytics and commenting can be achieved by integrating with existed online services.

Static site generators

Options are plenty, to name a few, Jekyll, Octopress, Pelican, Middleman, and Hugo. They were written in different languages (Ruby, JavaScript, Python and Go, etc.) but serve the same goal – construct the bare bone of a website and render the contents to the publishable HTML pages.

Not every site generators were created equal. Folks from different languages tend to choose the one they are familiar with. That was why I used Pelican initially. For the time-being, I realized that Pelican community is not very active, and theme options seems a bit thin. Jekyll is more matured, and there are already numerous professional-like themes available. However, it is Hugo that caught my eyes.

Hugo has certainly attracted many people from Jekyll. The binary itself is written in Go, and no other dependencies are necessary. It renders Markdown file to HTML really fast (we are talking about millisecond here.) Its structure makes switching between themes and customization overwriting the chosen theme super easy. Also, Hugo was the name I had during the 7-day intensive Spanish program that I completed in the past summer.

hugo

Hugo workflow

Let me walk you through the basics of building a Hugo-powered website. For more details, readers are referred to the official Hugo

Install Hugo binary: for OS X (and the future Mac OS) users, it’s recommended to do it through homebrew, just like for most open-source software.
```
 brew install hugo
```
Create a new site under the desired root directory:
```
 hugo new site sitename
```

The generated directory have the following layout:

 .
|-- archetypes
|-- config.toml
|-- content
|-- data
|-- layouts
`-- static
5 directories, 1 file

Create new posts under the content folder:
```
 hugo new post/postname.md
```
config.toml is the key configuration file to let Hugo know how to render the contents.
Pick a theme from or build your own. To apply a theme, one needs to create a themes folder under the root, and git clone or make a new-theme directory under this folder. Then switching themes can be achieved by adding a line in config.toml file like this theme = "hugo-academic", or tell hugo which theme to build with using this command hugo server --theme=hugo_theme_name
Check the results by running a virtual server, which automatically listens to any changes you make and render HTML in real time. Go to to see your shiny new site in action.
```
 hugo server --watch
```
Build the pages by invoking hugo. Generated HTML pages will by default be stored under public folder. It is this folder you need to find a home to host.

comments and tips

Posts are usually written in Markdown, though it also supports other markup languages. Markdown is really elegant and easy to use, definitely worth picking up.
The equivalent of extension in Hugo is shortcodes, which are stored under layout/shortcodes directory. These pre-defined snippets can allow site-owner to embed Youtube, Vimeo players and individual tweet into their posts.
Most themes support Disqus for commenting or google-analytic for analyzing website usage.
The format of metadata in the markdown files might be different from those using other site-generators. So, migrating old posts to Hugo needs some works (could be easily down with some simple parsing scripts.)
Add customized CSS stylesheet under static to overwrite settings in the chosen theme.

Renovate this place and new goals

Tue, 05 Jul 2016 16:06:00 +0000

I’ve just spent the long weekend updating my nearly-abandoned website. This year, I have passed my qualifying exam (February), and the paper I’ve been working on for almost an year is now finally accepted.

The website has been changed to a GitHub user-page, instead of a project-page. Also, now the website is automatically generated by Travis-CI, a very popular service recently, testing and deploying the codes to ensure portability. All the changes and new posts are made on my laptop. Html pages are built by Pelican/Python3, on the /source branch. A push to the remote host will trigger Travis-CI. With a little help from ghp-import, the Travis-CI-generated pages will be force-pushed to the /master branch. Voila, that is the website now you are looking at.

Through this little-over-two-days endeavor, I now had more knowledge on Github, Travis-CI, and more control on my own website templates. Now it’s just the matter of keeping things rolling – I made the resolution to post more frequently. In fact, I have just updated the Work page to include some of my recent presentations and abstracts. I will also post some of the notes from our group meeting.

Agulhas Group meeting notes, 2016-06-29

Mon, 04 Jul 2016 17:07:00 +0000

What scientific questions are the authors addressing?
How the phase of the SIOD and therefore the SST anomaly expression of the Indian Ocean, modulates the Southern Africa precipitation response to ENSO?

What methods do the authors use to address the questions?

SA Land area bound by the region 15S-35S; 12.5E-42.5E
Observation: Global Precipitation Climatology Project version 2.2 (GPCP) 1979 to 2014, 2.5 x 2.5 deg resolution. Hadley-NOAA Optimum Interpolation dataset (OISST) 1.0x1.0 deg
Three SST modes and their indices: ENSO (Nino3.4 >0.5K, table 2), IOD, Subtropical-IOD (Fig. 1)
Two experiments: (1) Forced by observed global SST (2) Forced by the leading pattern of glabal time-varying monthly SST anomaly added to the monthly climatology, to isolate the atmospheric response to ENSO. (EOF1, Fig. 4)
Each experiment has two sets of ensembles from two different models:(1) 30 members of ECHAM5.4, 50 members of GFS2, on 1x1 deg horizonal grid.
Test whether the SIOD modifies the relationship between SA precipitation and ENSO through an examination of SA precipitation as a function of SIOD phase in two forced simulations.

What are the authors’ main conclusions?

El Niño is associated with reduced precipitation in the Southern Africa, and La Niña tends to occur with abundant precipitation in the area. Both are further enhanced by out-of-phase SIOD, and disrupted by in-phase SIOD.
For the in-phase SIOD and ENSO during December-March, SA precipitation is associated with an SST anomaly over the Agulhas Current region.
ENSO is related to an equivalent barotropic Rossby wave over Southern Africa, that modifies the regional mid-tropospheric vertical motions and precipitation.
The EOF1 forced case, either phase of SIOD does not modulate the SA precipitation response to ENSO (indistinguishable Fig. 10 and 11)

What are the implications of the results?

ENSO has been an important parameter for rainy season SA precipitation prediction, as SST playing a critical role in the regional climate. The modulation relationship between SIOD and ENSO presented here can help to make more informed forecasts.

Why did you choose this paper? looking into papers about South African precipitation variability.

Who read this paper? Yu Cheng

Workflow of a CCSM4 run

Wed, 20 Apr 2016 00:00:00 +0000

Basic approach, startup run:

Create a new case using the script under ~/CCSM4/scripts, following the template: PATH, compset, resolution and machine file. Compset B is the fully coupled mode.
```
 ./create_newcase -case ~/ccsm4/$casename \
-compset B_2000 \
-res 0.9x1.25_gx1v6 \
-mach bluefire
```
Go to the $CASEROOT directory, `~/ccsm4/$case, modify the processor layout (env_mach_pes.xml) or env_conf.xml` files, then configure and build the model:
```
 ./configure -case
./$casename.$mach.build
```
Create Production test (Optional): This will clone the current case and create a parallel folder ../$casename_ERT.x$mach. Go to the folder, and execute following commands:
```
 ./$casename_ERT.$mach.build
bsub < $casename_ERT.$mach.test
cat TestStatus
```
Once pass the test run, go back to the $CASEROOT, modify the env_run.xml as well as %casename.$mach.run. Using the given xmlchange script is recommended. The %casename.$mach.run is the batch submit script. Depending on the machines, it could be executed through bsub or qsub. In the env_run.xml file, one can edit items such as STOP_N, STOP_OPTION and RESUBMIT, to adjust the simulation length, as well as the resubmit frequencies. This modifications don’t require rebuilding executables.

A branch or hybrid run:

Follow Step 1 in the previous example to create a new case $casename2.

Edit env_conf.xml:

 xmlchange -file env_conf.xml -id RUN_TYPE -val branch
xmlchange -file env_conf.xml -id RUN_REFCASE -val $casename1
xmlchange -file env_conf.xml -id RUN_REFDATE -val 0001-02-01

For a hybrid run, set RUN_TYPE to hybrid instead.

Configure and build the case executable.

 ./configure -case
./$casename.$mach.build

Copy the necessary restart/initial data into $RUNROOT, a path similar to .../exe/$casename2/run
Repeat Step 4 in the previous example.

–

File locking explained:

Locking variables is a feature of CCSM that prevents users from changing variables after they have been resolved (used) in other parts of the scripts system. The variables in env_case are locked when create_newcase is called. The env_conf and env_mach_pes variables are locked when configure is called. The env_build variables are locked when CCSM is built, and the env_run variables are never locked and can be changed anytime. In addition, the Macros file is locked as part of the build step. The $CASEROOT/LockedFiles directory saves copies of the xml files to facilitate the locking feature.

–

More advanced use cases can be found in Chapter 9 of the

Agulhas Group meeting notes, 2014-06-05

Fri, 05 Jun 2015 16:07:00 +0000

What scientific questions are the authors addressing?

To test the observed (by MOCHA array) 8 year trend (2004-2012, -0.53Sv/yr) of AMOC is significant, in comparison to the 14 CMIP5 models preindustrial control runs.

What methods do the authors use to address the questions?

The existed oberserved timeseries is not long enough, and there is yet no consensus on the magnitude, periodicities, and mechanisms of AMOC variability across different models.

Pick out the models with overturning streamfunction diagnostics on the CMIP5 archive (14), indices defined as the maximum streamfunction at the model latitude closest to 26.5N, annual mean taken from Apr-Mar adopted by the RAPID array (why?)
Simplely compare distribution of 8yr trends of different models to the observed trend. However, every models seem underestimate the interannual variability of AMOC (Fig2, b,c)
Combining the temporal correlation characteristics of AMOC from CMIP5 models with an observational estimate of interannual variability to account for this bias. (using autoregressive moving average model to fit CMIP5 AMOC timeseries, and add the upper limit of observed variability.)

What are the authors’ main conclusions?

Eight year trends ≤ −0.53 Sv/yr are relatively common in GFDL-CM3 and GFDL-ESM2M, but they are extremely unusual (or out of range) in the other 12 models.
Applying modification of observed variability on top of temporal correlation of Model AMOC timeseries, the observed AMOC trend is not significantly different (p > 0.01) from that expected due to internal AMOC variability. (Fig 2, 3)
If the AMOC continues to decrease at a rate of −0.53 Sv/yr, it will take a total of 18 yearsbefore the trend is significantly different from all 14 ARMA-based estimates of internal AMOC variability (red line in Figure 3).

What are the implications of the results?

Authors find that 14 different CMIP5 models underestimate the magnitude of AMOC variability on interannual time scales, which may due to insufficient atmospheric variability on interannual time scales, and the coarse resolution of ocean in CMIP5 simulations.
It is not yet possible to distinguish between mechanisms or dominant time scales due to the short length of the existing observational AMOC record. Discriminating between different models and refining estimates of internal AMOC variability require more than one decade of continuous AMOC observations. All this is calling for the consruction of a more powerful observation system, including monitoring water mass properties in the subpolar gyre and similar arrays at additional lattitudes.

Why did you choose this paper? I am digging out materials about AMOC in climate models.

Who read this paper? Yu Cheng

Agulhas Group meeting notes, 2014-07-10

Thu, 17 Jul 2014 16:50:00 +0000

What scientific questions are the authors addressing?

The maximum strength of Agulhas leakage is identified during the glacial Termination II. The authors attempt to demonstrate the leakage, mostly in the form of rings, was effectively incorporated into the South Atlantic gyre, instead of retroflecting back to Southern or Indian ocean.

What methods do the authors use to address the questions?

They investigate the presence of paleo Agulhas rings from a sediment core on the central Walvis Ridge (64PE-174P13, 29.45.71’S, 2.24.10’E, 2914m deep). By selecting the appropriate species of plaktic foraminifer, they aim to target the salinity and temperature anomalies induced by the leakage at the depth of the thermocline.
Analyzing INALT01 60yr model output to relate density perturbations at thermocline depth to the passage of individual rings over the core site. They concluded that the increasing variability of density at thermocline depth can be contributed to increasing passage of Agulhas rings.
Using this relation as a basis of proxy of Agulhas leakage, they generate a time series of individual Globorotalia truncatulinoides O_18 variability to infer the variability of Agulhas leakage.

What are the authors’ main conclusions?

Strong relation between SSH and potential density at 452m variability is prominent in INALT01. Fig3 shows the temporal variability induced by individual Agulhas ring crossings the chosen core location. The largest density troughs can be attributed to passing Agulhas rings. Out of average 6 rings/year, the core site captures on average one to two full rings according to a Fourier analysis of SSH on the location.
The time series shows high levels of pycnocline depth variability at the site, suggesting enhanced numbers of Agulhas rings moving into the South Atlantic Gyre around glacial Termination II.
Their results closely follow the previous quantifications of Agulhas leakage from the east of the Cape Basin (other sediments cores.), showing that Indian Ocean waters enter the South Atlantic circulation.

What are the implications of the results?

By targeting the isotopes in appropriate species, scientists can not only study certain era of sediments cores, but also specific levels of the ocean.
This result is a crucial support for the view of a prominent role of the Agulhas leakage in the shift from a glacial to an interglacial mode of the Atlantic circulation.
Even though the increased leakage during the glacial termination did make it into subtropical gyre, the corresponding salt and heat were not neccesarily advected across the equator. However, modeling evidence show that the heat and salt from the Leakage do not have to reach the North Atlantic to impact the AMOC.

Why did you choose this paper? Back to work on leakage, digging something more that supporting the claim that Agulhas leakage has impact on Atlantic circulation.

Who read this paper? Yu Cheng

Building a static blog with Pelican

Tue, 10 Jun 2014 12:05:00 +0000

Once upon a time…

I spent a summer using Microsoft Frontpage to build a website, hosted by my highschool’s server. Then I also had my first experience with PHP, javascript, html, css, cgi (common gateway interface, I haven’t seen such things for a while!) However, some popular free-webpage services just kept popping out the years after, as “blog” became a everyday word. All of a sudden, a vibrant and powerful “blog”, usually implemented with guestbook and albums is just few clicks away. Thus, I stop writing my own webpages. That was ten years ago. Years later, even MS Frontpage is discontinued. End of last summer, every students in RSMAS got an mail about annual progress report, and building your personal page is listed as a requirment. I first cheated by adding a new blog using wordpress, refusing to revisit my old friends, being afraid of spending too much time on such distractions. However, these distractions just won’t dispear even though I tried hard to ignore… Therefore, after the tiring semster with TA and seminar talk, I came back to revisit some other interesting stuffs I’ve had no time to spend on for the past few months.

Pelican: I love this bird!!

I’ve heard about Django quite often, so it was my first try. Lacking of PHP knowledge, and a personal site needs not to have some log-in or user-management functions, I decided to build a static blog. By accident, I noticed that lots of technical blogs are hosted by Github, and powered by Jekyll, Octopress or Pelican. The reason that I chose Pelican, is simply because it’s python/jinja2 based. And I happened to find some very good references:

{% img http://www.allaboutbirds.org/guide/PHOTO/LARGE/brown_pelican_5.jpg 400 500 %}

Overview

Pelican is a static site engine powered by python, using templates based on Jinja2. It supports both reStructuredText and Markdown, as well as asciiDoc and html, which makes writing new posts extremely easy. The center of a pelican-powered site is the configuration.py file, which controls the basic settings of your site, including side bar, navigation bar options, disqus-comments, tags, categories, twitter supports, and PATH. All the original files (including .md, .rst files) are stored in /contents, and all the html generated by Pelican go to /output folder. Simply upload the /output folder to your web-host, and T’ADA, your website is on. For details of installation and deployment, pleace check official Pelican documentation.

Themes

One of the reasons that I chose Pelican is the active community on Github. As mentioned above, pelican templates are generated by Jinja2, which is a template engine powered by Python. A set of themes including templates of different pages and stylesheets. There are many high-quality themes available on github, which can be easily downloaded using following command:

:::bash
git clone https://github.com/getpelican/pelican-themes

This blog is powered by pelican-bootstraps3 theme; few other themes I would also like to try are Pelican-pure, Mockingbird and Notmyidea. Additional stylesheet can be easily integrated by changing the setting in configuration.py. To switch between different themes, just modify this line in your configuration.py.

:::python
THEME = "pelican-bootstrap3" # or whatever themes you like

Customize

This is a more tricky part, due to my limited knowledge of CSS and Jinja2. So far I’ve only managed to change the background color, font size, line-height and link-hover style of my header. Even though the syntax of CSS is very straightforward, lacking of nested structure, it is very difficult to read the CCS stylesheet by someone else: one must figure out the class/ID names of each elements in the html files, and locate corresponding tags in the minified CSS file. I’ve found a Sublime-Text package to unminify the CSS, yet still find it difficult to change even a single property. Currently, I am attempting to use SASS/COMPASS to rewrite the stylesheet.

Plugins

One of greatest feature of Pelican is to include modular plugins into your static site. Same as Pelican-themes, these plugins can be downloaded easily from github.

:::bash
git clone https://github.com/getpelican/pelican-plugins.git

Curretnly, I am using the Liquid-tags plugin by to enable Youtube video, iPython notebook, code-block and image insertion in my blog posts, as well as the Math-rendering. Other plugins such as Gravatar and internal-links are very easily to be implemented, in fact, just two lines in the configuration.py. The idea of plugins, are similar to the idea of importing libraries in the beginning of every Python scripts.

:::python
# Plugins
PLUGIN_PATH = '/Users/yucheng/Desktop/Other/MyPage/pelican-plugin'
PLUGINS = ['liquid_tags.img', 'liquid_tags.video',
'liquid_tags.youtube', 'liquid_tags.vimeo',
'liquid_tags.include_code', 'liquid_tags.notebook','render_math']

Summary

Pelican is a really powerful static-site generator, using Python. All options you may play with are in the configuration.py and the syltesheet.css. The official documentation of Pelican is rather complete, not to mention the active developer-base. If you are a Python lover, it won’t take you too long to setup a really polished and somehow-dynamic static site. So, look no further, among numerous static-site engines, Pelican is definitely one of the best.

Posts |

Supercharging Your Job Hunt with ChatGPT and Espanso

Job Searching with ChatGPT

Use Espanso to speed things up

Getting Started with Espanso

Sample triggers

Wildfire Smoke is Choking Seattle: visualizing using Altair

https://airnow.gov/index.cfm?action=aqibasics.aqi

Wildfires and climate change

Parsing a .xlsx file generated by Labchart

Step 1: Data inspection

Step 2: Data cleaning

Step 3: Plotting/Groupby/Processing

Remove model drift using control run simulations

HRC07 TS timeseries at 40E, 45S, and the cubic fit from control runs.

HRC07 TS linear trends from 1953 to 2008 with model drift removed

Updated Hugo Academic theme to sync the upstream repo

Emojis!!!

Blockquotes

Syntax highlighting using highlight.js

Twitter Tweet

Running wild with @BearGrylls pic.twitter.com/RG3pRquOuu— Roger Federer (@rogerfederer) February 19, 2017

Youtube video

Alerts

Using git-latexdiff to track changes in tex-file graphically

Git basics

Track changes

Fill in the missing data using Python pandas

Working with missing data

Atlantic OHC in multiple layers 1951-2002

filled missing data with linear interpolation

Gibbs SeaWater Oceanographic Package for Python

Jupyter notebook gallery

Satellite SST and onboard observed wind-vectors

Remove model drift using control run simulations

More like overreactions?###

How to read the NWS hurricane warning?

NWS hurricane warning

forecast path

force wind affected area

force wind speed probabilities

Others

Watches vs Warnings

Automated deployment of Hugo generated site to Github pages

Basic setting in my config.toml for the Academic Theme

Github pages, user site or project site

Automated deployment using Wercker

Wercker dashboard: to find existed steps, click the Registry tab; to create your own applications and steps, access the Create tab.

Build

Deploy

Click deploy pipeline to setup included environment variables. Get the Github access token from Github profile setting.

Switch from Pelican to Hugo

Static site generators

Hugo workflow

comments and tips

Renovate this place and new goals

Agulhas Group meeting notes, 2016-06-29

Workflow of a CCSM4 run

Basic approach, startup run:

A branch or hybrid run:

File locking explained:

Agulhas Group meeting notes, 2014-06-05

Agulhas Group meeting notes, 2014-07-10

Building a static blog with Pelican

Once upon a time…

Pelican: I love this bird!!

Overview

Themes

Customize

Plugins

Summary

Running wild with @BearGrylls pic.twitter.com/RG3pRquOuu
— Roger Federer (@rogerfederer) February 19, 2017