The odd symbols come from a character encoding mismatch between the old and new database. Find which setting changed, repair the stored text once on a copy, then check pages, forms and emails.

Odd symbols across the site

The site moved to a new host, or its database was exported and imported again. The next morning, apostrophes all over the site have turned into strings such as ’.

French accents fare no better: café comes out as café, and some letters become a black diamond with a question mark inside.

Plain words are untouched, so the damage is patchy. A page of straightforward English looks fine, while the menu, the testimonials and anything pasted from Word look broken.

Nothing has been deleted. The text is still there, being read with the wrong table of letters.

Why it happens

Computers store each letter as a number, and a character encoding is the table that says which number means which letter. Plain English letters have the same numbers in every common table.

Accented letters, curly quotes and symbols do not. Older tables such as Latin-1 store an accented e as one number; UTF-8, the current standard, stores it as two.

Text saved in one encoding and read in the other falls apart. Each two-number letter shows as two wrong letters, and a curly apostrophe, three numbers in UTF-8, becomes three.

That is why the straight apostrophe typed on a keyboard survives. It is plain text in every table. The curly one that Word substitutes is not.

Where the mismatch hides

The second is the usual culprit after a move. Many older sites wrote UTF-8 text into tables labelled Latin-1, over a connection set the same way, so it came back out unchanged and looked fine.

The export then treated that text as Latin-1 and converted it to UTF-8 a second time. On the new server, read correctly, that doubled conversion is what shows.

Do not start with search and replace

The obvious repair is a search and replace across the database. It fixes the commonest strings and misses the rest, since every accented letter has its own garbled form.

Done carelessly, it can also break settings that WordPress and many other systems store with the length of each piece of text recorded beside it. Change the length and the setting stops loading.

And it leaves the cause in place, so the next export repeats the damage. Find the setting first, then repair the text once.

Fixing the setting

Start on the old server, if it still exists. Its tables show which encoding they were labelled with, and the site's configuration shows which one the connection used.

In WordPress that is a line in the configuration file naming the database character set. Hand-built sites set it in the code that opens the database connection, or leave it out and take the server's default.

The pages should then declare UTF-8, and the server's own header should agree with them. When the two disagree, browsers believe the server.

WordPress 4.2, released yesterday, adds emoji. Where the server allows, its update converts UTF-8 tables to a fuller form, utf8mb4, that can store them. Tables labelled Latin-1 are left alone, and so is text already garbled.

Repairing the stored text

Doubled text can be converted back cleanly. Export it again from the old server with the connection set to Latin-1, so nothing is converted on the way out, then import that file into tables set to UTF-8.

Take a full backup first and do the work on a copy. Then compare a dozen pages with accents, quotes and symbols against the old site before switching over.

Pages edited since the move need care. They now mix new, correct text with old, garbled text, and a blanket conversion would garble the new part. Note every page someone has touched.

A worked example

A family-run Peruvian restaurant in a suburban strip mall lists its whole menu on its website, with each dish named in Spanish and described in a line of English.

In March the site moved to a new host. The next morning causa limeña read as causa limeña, the note about the family's recipes was full of ’, and reservation emails arrived from guests named José.

A family member corrected a few dishes by hand in the editor. That tidied the menu and made the real repair harder, because those pages now held a mixture of good and bad text.

The developer found the old tables labelled Latin-1 but holding UTF-8. He exported them again with that setting, imported them into UTF-8 tables, redid the hand-edited pages, and made the reservation emails declare their encoding.

Forms, emails and exports

Check the places where text leaves the site. A contact form email that does not declare UTF-8 turns an accented name into nonsense, and the reply goes out addressed that way.

Order confirmations, newsletters and anything built from the database follow the same rule. Send a test through each with an accented name and an apostrophe pasted from Word.

A customer list saved as a CSV file can also open in Excel with every accent garbled, even when the file is correct. Excel's text import, set to UTF-8, shows it properly.

Search results show it too

Page titles and descriptions carry the same garbled characters, and Google keeps showing them in its results until it visits those pages again.

Once the repair is live, use Fetch as Google in Webmaster Tools on the main pages and submit them to the index, which asks Google to look again sooner.

The counter-case

Not every odd character means a database problem. One page with a few broken quotes usually had text pasted from Word or an email, and correcting that page in the editor is the sensible repair.

A small site built from plain files, without a database, can likewise be fixed page by page, provided each file is saved as UTF-8 and says so.

When the symbols appear across the whole site at once, after a move or an export, treat it as a setting and repair it once, on a copy.

After a move

  1. Stop editing pages until it is fixed.
  2. Check the old tables and connection setting.
  3. Back up the database before changing anything.
  4. Repair the text once, on a copy.
  5. Declare UTF-8 in pages, headers and emails.
  6. Send a test enquiry with an accented name.
  7. Ask Google to recrawl the main pages.

Step one is the one to do today. Every page edited in the meantime mixes good text with garbled text, and each one turns part of a single clean repair into work by hand.

Old code meeting a new server causes a related problem, covered in the deprecated warning across the top of the page.

Frequently asked questions

Why do apostrophes turn into odd symbols?

The text was saved in one character encoding and is being read in another. Curly apostrophes and accented letters are stored differently in each, so they fall apart.

Why are only some apostrophes broken?

A straight apostrophe typed on a keyboard is plain text in every encoding. The curly one Word substitutes is not, so only text pasted from Word breaks.

Can I just search and replace the symbols?

It catches the commonest cases and misses the rest, can break settings stored with their length, and leaves the cause in place for the next export.

What usually causes it after a move?

Older sites often stored UTF-8 text in tables labelled Latin-1. The export converts it a second time, and the new server shows the doubled result.

Should I fix pages by hand meanwhile?

No. Edited pages mix correct and garbled text, which a single clean repair can no longer handle. Stop editing until the database has been repaired.

What should I check after the repair?

Form emails, order confirmations, newsletters and spreadsheet exports, each tested with an accented name. Then ask Google to recrawl the main pages through Webmaster Tools.

West Coast Media Solutions Inc. provides web design, web development, hosting, digital marketing, and business consulting to organisations across Canada, drawing on more than twenty-five years in the field.

Apostrophes garbled after a move?

Stop editing pages and tell us where the site was hosted before. We will repair the text once, properly.

Start a Conversation