一、chunking 句子分块
segmentation and labeling at both the token and chunk levels
1 noun phrase chunking( np-chunking)名词短语
import nltk
sentence = [("the", "dt"), ("little", "jj"), ("yellow", "jj"),("dog", "nn"), ("barked", "vbd"), ("at", "in"), ("the", "dt"), ("cat", "nn")]
grammar = "np: {
?*}"
cp = nltk.regexpparser(grammar)
result = cp.parse(sentence)
print(result)
result.draw()
----------------------------------------------------------
如果觉得上述对句子人式标注很麻烦，可以改为以下：
import nltk
sentence="the little dog baked at the cat"
words=nltk.word_tokenize(sentence) #分词
sent_tag=nltk.pos_tag(words) #加标注
grammar = "np: {
?*}"cp = nltk.regexpparser(grammar)
result = cp.parse(sent_tag)
print(result)
result.draw()
练习1：考虑下面的名词短语如何表示。
another/dt sharp/jj dive/nn
trade/nn figures/nns
any/dt new/jj policy/nn measures/nns
earlier/jjr stages/nns
panamanian/jj dictator/nn manuel/nnp noriega/nnp二、句子结构分析
1 句子嵌套
usain bolt broke the 100m recordb.
the jamaica observer reported that usain bolt broke the 100m recordc.
andre said the jamaica observer reported that usain bolt broke the 100m recordd.
i think andre said the jamaica observer reported that usain bolt broke the 100m record
2 句子歧义
i shot an elephant in my pajamas.http://www.nltk.org/book/ch08.html
groucho_grammar = nltk.cfg.fromstring("""
s -> np vp
pp -> p np
np -> det n | det n pp | 'i'
vp -> v np | vp pp
det -> 'an' | 'my'
n -> 'elephant' | 'pajamas'
v -> 'shot'
p -> 'in'
""")
sent = ['i', 'shot', 'an', 'elephant', 'in', 'my', 'pajamas']
parser = nltk.chartparser(groucho_grammar)
for tree in parser.parse(sent):
print(tree)
3 context free grammar 上下文无关文法
(1)递归下降解析器
nltk.app.rdparser()
(2)移进-归约解析器
nltk.app.srparser()
评论 0