[Java]SiteMap Creater-簡單的SiteMap建立程式

SiteMapCreater主要的功能在自動產生SiteMap.xml,只需要在資料庫表格裡建立所要產生的SiteMap網址等資訊就可以執行它來自動產生需要的SiteMap而提交給Google或yahoo等搜尋引擎。

這個程式主要是獨立作業的,可以利用windows或是linux的排程來執行jar run就可以了,這樣子可以減少對伺服器的負擔。

程式的作法是從資料庫取得資料,再把資料加上xml需要的字串,然後最後輸出成檔案就可以了,中間有些需要注意的地方,像是筆數不可太多,此程式設定4000筆會換一個檔案名稱,還有檔案也不可超過太大,8mb以下或更小。

執行後會得到以下檔案,一個index的Sitemap.xml及實際的內容的Info0.xml,另外還需要一個xsl來當讀取樣版的準則,向搜尋引擎只需要提交Sitemap.xml這個檔案就可以了,程式還會自動把所有檔案變成一個zip檔,google可以提交zip檔案做為SiteMap,不過此部份並未成功,有興趣的朋友可以再加以改進。

SiteMap1.png




主要產生xml程式碼

package yku.xml;

public class SiteMapCreator {
    private org.apache.log4j.Logger logger;

    private java.text.DateFormat dformat;

    private String directory;// 存放SiteMap xml 路徑

    private String Url_Directory;// SiteMap url路徑

    private String ZipToFile;// 做成Zip的目錄及名稱

    private java.util.ArrayList SiteMapList;

    private String Header;

    private String End;

    public SiteMapCreator() {
        logger = org.apache.log4j.Logger.getLogger(this.getClass());
        dformat = new java.text.SimpleDateFormat("yyyy-MM-dd HH:mm");
        SiteMapList = new java.util.ArrayList();
        directory = yku.setup.P.getV("Directory");
        Url_Directory = yku.setup.P.getV("Url_Directory");
        ZipToFile = yku.setup.P.getV("ZipToFile");
        Header = "<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
                + "<?xml-stylesheet type=\"text/xsl\" href=\"wa_google.xsl\"?>"
                + "<urlset xmlns:xsi=\"http://www.w3.org/2001/XMLSchema"
                + "-instance\" xsi:schemaLocation=\"http://www.sitemaps."
                + "org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas"
                + "/sitemap/0.9/sitemap.xsd\" xmlns=\"http://www.sitemaps."
                + "org/schemas/sitemap/0.9\">";

        End = "</urlset>";
        java.io.File f = new java.io.File(directory);
        if (!f.exists())
            f.mkdir();

        // 建立xsl,如果xsl存在copy一份過去
        f = new java.io.File(f.getAbsoluteFile() + "/wa_google.xsl");

        if (!f.exists()) {
            java.io.File xsl = new java.io.File("wa_google.xsl");
            copyfile(xsl, f);

        }
    }

    public void run() {
        MakeSiteMap();
        outIndex();
        zip();
    }

    private String XmlElement(String url, String time, String changefreq,
            double priority) {
        return "<url><loc>" + url + "</loc><lastmod>" + time
                + "</lastmod><changefreq>" + changefreq
                + "</changefreq><priority>" + priority + "</priority></url>";
    }

    private void MakeSiteMap() {
        int count = 0;// 計算輸出了幾筆
        String sql = "select * from sitemapinfo";
        java.sql.ResultSet rs = null;
        java.sql.Statement stat = null;
        java.sql.Connection con = null;

        java.io.PrintWriter fw = null;
        try {
            con = yku.database.Manager.getConnection();
            stat = con.createStatement();
            rs = stat.executeQuery(sql);
            while (rs.next()) {
                if (count % 40000 == 0) {
                    if (fw != null) {
                        fw.println(End);
                        fw.flush();
                        fw.close();
                        fw = null;
                    }
                    // SiteMap的檔案名稱
                    logger.info("File Name:" + directory + "Info"
                            + (count / 40000) + ".xml");
                    java.io.File f = new java.io.File(directory + "Info"
                            + (count / 40000) + ".xml");
                    SiteMapList.add("Info" + (count / 40000) + ".xml");

                    fw = new java.io.PrintWriter(f, "UTF-8");
                    fw.println(Header);
                }
                try {
                    java.sql.Timestamp time = rs.getTimestamp("time");
                    fw.write(XmlElement(Url_Directory + rs.getString("url"),
                            dformat.format(time), rs.getString("freq"), rs
                                    .getDouble("priority")));
                    count++;
                } catch (java.sql.SQLException e) {
                    e.printStackTrace();
                    logger.info("MakeSiteMap:" + "處理錯誤:" + e);
                }
            }
            fw.println(End);
            fw.flush();
            fw.close();

        } catch (java.sql.SQLException e) {
            e.printStackTrace();
            logger.info("MakeSiteMap:" + e);
        } catch (java.io.IOException e) {
            e.printStackTrace();
            logger.info("MakeSiteMap:" + e);
        } finally {
            try {
                if (rs != null) {
                    rs.close();
                    rs = null;
                }
                if (stat != null) {
                    stat.close();
                    stat = null;
                }
            } catch (java.sql.SQLException ex) {
                ex.printStackTrace();
                logger.info("MakeSiteMap:" + ex);
            }
            try {
                if (con != null) {
                    con.close();
                    con = null;
                }
            } catch (java.sql.SQLException e) {
                e.printStackTrace();
                logger.info("MakeSiteMap:" + e);
            }
        }

    }

    private void outIndex() {
        java.io.PrintWriter fw = null;
        try {
            java.io.File f = new java.io.File(directory + "Sitemap.xml");
            fw = new java.io.PrintWriter(f, "UTF-8");
            fw
                    .println("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\r\n"
                            + "<sitemapindex xmlns:xsi=\"http://www.w3.org/2001/"
                            + "XMLSchema-instance\" xsi:schemaLocation=\"http:"
                            + "//www.sitemaps.org/schemas/sitemap/0.9 http://www"
                            + ".sitemaps.org/schemas/sitemap/0.9/sitemap.xsd\" "
                            + "xmlns=\"http://www.sitemaps.org/schemas/sitemap/0.9\"> ");
            for (int i = 0; i < this.SiteMapList.size(); i++) {
                fw
                        .println("<sitemap><loc>"
                                + Url_Directory
                                + SiteMapList.get(i)
                                + "</loc><lastmod>"
                                + dformat.format(new java.util.Date(System
                                        .currentTimeMillis()))
                                + "</lastmod></sitemap>");
                fw.flush();
            }
            fw.println("</sitemapindex>");
            fw.flush();
            // this.SiteMapList.add("Sitemap.xml");
        } catch (java.io.IOException e) {
            e.printStackTrace();
            logger.info("outIndex:" + e);
        } finally {
            if (fw != null) {
                try {
                    fw.close();
                } catch (Exception e) {
                    e.printStackTrace();
                    logger.info("outIndex:" + e);
                }
                fw = null;
            }

        }
    }

    // 壓縮成zip檔
    private void zip() {

        java.io.File outFile = new java.io.File(this.ZipToFile);
        java.util.zip.ZipOutputStream zo = null;
        java.io.BufferedOutputStream bs = null;
        java.io.FileOutputStream fos = null;
        java.io.FileInputStream fis = null;
        try {
            fos = new java.io.FileOutputStream(outFile);
            bs = new java.io.BufferedOutputStream(fos);
            zo = new java.util.zip.ZipOutputStream(bs);
            byte[] buf = new byte[1024];
            int len;
            // System.out.println("baseFile: "+baseFile.getPath());
            for (int i = 0; i < this.SiteMapList.size(); i++) {
                fis = new java.io.FileInputStream(this.directory
                        + this.SiteMapList.get(i));
                // 建立檔案名稱
                zo.putNextEntry(new java.util.zip.ZipEntry(
                        (String) this.SiteMapList.get(i)));
                while ((len = fis.read(buf)) > 0) {
                    zo.write(buf, 0, len);
                }
                fis.close();
            }
        } catch (java.io.IOException e) {
            logger.info("Zip Error:" + e);
        } finally {
            try {
                if (zo != null) {
                    zo.close();
                    zo = null;
                }
                if (bs != null) {
                    bs.close();
                    bs = null;
                }

                if (fis != null) {

                    fis.close();

                    fis = null;
                }
                if (fos != null) {
                    fos.close();
                    fos = null;
                }

            } catch (java.io.IOException e) {

            }
        }
    }

    private void copyfile(java.io.File f1, java.io.File f2) {
        try {

            java.io.InputStream in = new java.io.FileInputStream(f1);

            // For Append the file.
            // OutputStream out = new FileOutputStream(f2,true);

            // For Overwrite the file.
            java.io.OutputStream out = new java.io.FileOutputStream(f2);

            byte[] buf = new byte[1024];
            int len;
            while ((len = in.read(buf)) > 0) {
                out.write(buf, 0, len);
            }
            in.close();
            out.close();
            logger.info("XLS File copied.");
        } catch (java.io.FileNotFoundException ex) {
            logger.info(ex.getMessage() + " in the specified directory.");
            System.exit(0);
        } catch (java.io.IOException e) {
            logger.info(e.getMessage());
        }
    }

    public static void main(String args[]) {
        (new SiteMapCreator()).run();
    }
}

連結資料庫程式碼

利用讀取外部設定檔來取得mysql ConnectionPool,再向此Pool取得java.sql.Connection

package yku.database;

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.util.Properties;

import javax.sql.DataSource;

public class Manager {
    public static com.mysql.jdbc.jdbc2.optional.MysqlDataSource ds = null;

    public static DataSource getPool() {
        // com.mysql.jdbc.jdbc2.optional. ds = null;

        if (properties == null)
            loadJDBCProperties();
        // ds = new oracle.jdbc.pool.OracleConnectionPoolDataSource ();
        if (ds == null) {
            ds = new com.mysql.jdbc.jdbc2.optional.MysqlDataSource();

            // 傳入參數
            ds.setServerName(properties.getProperty("ServerName"));
            ds.setPortNumber(Integer.parseInt(properties.getProperty("port")));
            ds.setDatabaseName(properties.getProperty("DatabaseName"));
            ds.setUser(properties.getProperty("user"));
            ds.setPassword(properties.getProperty("password"));
        }

        return ds;
    }

    public static java.sql.Connection getConnection()
            throws java.sql.SQLException {
        return getPool().getConnection();
    }

    private static java.util.Properties properties = null;

    private static void loadJDBCProperties() {
        // create JDBC.properties file handle
        File file = new File("jdbc.properties");
        file = new File(file.getAbsolutePath()); // for Windows bug

        yku.log.out.println("ConnectionManager:load JDBC properties: "
                + file.getAbsolutePath());

        try {
            FileInputStream input = null;
            try {
                input = new FileInputStream(file);
                properties = new Properties();
                properties.load(input);
            } finally {
                if (input != null) {
                    input.close();
                }
            }
        } catch (IOException ex) {
            throw new RuntimeException(ex);
        }
        file = null;

    }
}

設定檔如下,名稱jdbc.properties

需放至與classes同目錄(或jar同目錄)

ServerName=localhost
DatabaseName=test
port=3306
user=root
password=12345

讀取外部參數檔成設定值

package yku.setup;

import java.io.IOException;

public class P {
    private static java.util.Properties p = null;

    public static String getV(String value) {
        if (p.containsKey(value))
            return p.getProperty(value);
        else
            return "";
    }

    static {
        try {
            // 讀取properties檔
            java.io.File f = new java.io.File("default.properties");

            java.io.InputStream is = new java.io.FileInputStream(f);

            p = new java.util.Properties();

            p.load(is);

            is.close();
            java.util.Iterator itr = p.entrySet().iterator();

            while (itr.hasNext()) {
                Object key = itr.next();
                yku.log.out.print("預設值:" + key);
            }
        } catch (IOException ie) {
            ie.printStackTrace();

        }

    }
}

外部設定檔名稱default.properties

跟jdbc.properties放至相同目錄就可以了

Directory是xml產生目錄,Url_Directory是主要的DNS名稱,

ZipToFile是Zipe要存放的位置及名稱,SQL是向資料庫取得內容的語法,

FiledUrl FileTime FileFreq FilePriority是取得的欄位名稱

SubXmlName是xml的檔案名稱前致詞

Directory = d:/SiteMap/
Url_Directory=http://blog.yslifes.com
ZipToFile=d:/SiteMap/SiteMap.xml.zip
Sql = select * from sitemapinfo
FieldUrl = url
FieldTime = time
FieldFreq = freq
FieldPriority = priority
SubXmlName = Info

範例Sql如下:

CREATE TABLE  `sitemapinfo` (
  `id` int(10) unsigned NOT NULL auto_increment,
  `url` varchar(500) NOT NULL,
  `time` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
  `freq` varchar(45) NOT NULL,
  `priority` double NOT NULL,
  PRIMARY KEY  (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

INSERT INTO sitemapinfo(url,time,freq,priority) values('/archives/620',DATE_FORMAT('20100305','%Y%m%d'),'Monthly',20);
INSERT INTO sitemapinfo(url,time,freq,priority) values('/archives/619',DATE_FORMAT('20100304','%Y%m%d'),'Weekly',100);
INSERT INTO sitemapinfo(url,time,freq,priority) values('/',DATE_FORMAT('20100301','%Y%m%d'),'Daily',30);
INSERT INTO sitemapinfo(url,time,freq,priority) values('/archives/basic',DATE_FORMAT('20100302','%Y%m%d'),'Yearly',70);

更詳細的內容可以下載原始碼檔案程式

One thought to “[Java]SiteMap Creater-簡單的SiteMap建立程式”

發表迴響