在Python中从字符串中剥离除字母数字字符以外的所有内容

使用Python从字符串中剥离所有非字母数字字符的最佳方法是什么?

这个问题的PHP变体中提出的解决方案可能会进行一些小的调整，但对我来说似乎不是很“python化”。

声明一下，我不只是想去掉句号和逗号(以及其他标点符号)，还想去掉引号、括号等。

476621 次浏览
小开

正则表达式的拯救:

import re
re.sub(r'\W+', '', your_string)

通过Python定义'\W == [^a-zA-Z0-9_]，它排除了所有numbers， letters和_

小开

如何:

def ExtractAlphanumeric(InputString):
from string import ascii_letters, digits
return "".join([ch for ch in InputString if ch in (ascii_letters + digits)])

它的工作原理是，如果ascii_letters和digits字符串组合中存在InputString中的字符，则使用列表推导生成一个字符列表。然后它将列表连接在一起，形成一个字符串。

小开 
>>> import re
>>> string = "Kl13@£$%[};'\""
>>> pattern = re.compile('\W')
>>> string = re.sub(pattern, '', string)
>>> print string
Kl13
小开

你可以试试:

print ''.join(ch for ch in some_string if ch.isalnum())
小开
最佳答案

我只是出于好奇计算了一些函数的时间。在这些测试中，我从字符串string.printable(内置string模块的一部分)中删除非字母数字字符。使用编译后的'[\W_]+'和pattern.sub('', str)被发现是最快的。

$ python -m timeit -s \
"import string" \
"''.join(ch for ch in string.printable if ch.isalnum())"
10000 loops, best of 3: 57.6 usec per loop


$ python -m timeit -s \
"import string" \
"filter(str.isalnum, string.printable)"
10000 loops, best of 3: 37.9 usec per loop


$ python -m timeit -s \
"import re, string" \
"re.sub('[\W_]', '', string.printable)"
10000 loops, best of 3: 27.5 usec per loop


$ python -m timeit -s \
"import re, string" \
"re.sub('[\W_]+', '', string.printable)"
100000 loops, best of 3: 15 usec per loop


$ python -m timeit -s \
"import re, string; pattern = re.compile('[\W_]+')" \
"pattern.sub('', string.printable)"
100000 loops, best of 3: 11.2 usec per loop
小开

使用str.translate()方法。

假设你会经常这样做:

  1. 一次，创建一个包含所有你想删除的字符的字符串:

    delchars = ''.join(c for c in map(chr, range(256)) if not c.isalnum())

  2. 当你想要压缩字符串时:

    scrunched = s.translate(None, delchars)

设置成本可能比re.compile;边际成本更低:

C:\junk>\python26\python -mtimeit -s"import string;d=''.join(c for c in map(chr,range(256)) if not c.isalnum());s=string.printable" "s.translate(None,d)"
100000 loops, best of 3: 2.04 usec per loop


C:\junk>\python26\python -mtimeit -s"import re,string;s=string.printable;r=re.compile(r'[\W_]+')" "r.sub('',s)"
100000 loops, best of 3: 7.34 usec per loop

注意:使用__ABC0作为基准数据会给模式'[\W_]+'带来不公平的优势;所有的非字母数字字符都在一堆…在典型的数据中，会有不止一个替换:

C:\junk>\python26\python -c "import string; s = string.printable; print len(s),repr(s)"
100 '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&\'()*+,-./:;=>?@[\\]^_`{|}~ \t\n\r\x0b\x0c'

下面是如果你给re.sub更多的工作，会发生什么:

C:\junk>\python26\python -mtimeit -s"d=''.join(c for c in map(chr,range(256)) if not c.isalnum());s='foo-'*25" "s.translate(None,d)"
1000000 loops, best of 3: 1.97 usec per loop


C:\junk>\python26\python -mtimeit -s"import re;s='foo-'*25;r=re.compile(r'[\W_]+')" "r.sub('',s)"
10000 loops, best of 3: 26.4 usec per loop
小开

作为这里其他一些答案的衍生，我提供了一种非常简单而灵活的方法来定义您想要限制字符串内容的一组字符。在这种情况下，我允许字母数字加破折号和下划线。只要添加或删除字符从我的PERMITTED_CHARS适合你的用例。

PERMITTED_CHARS = "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ_-"
someString = "".join(c for c in someString if c in PERMITTED_CHARS)
小开 
for char in my_string:
if not char.isalnum():
my_string = my_string.replace(char,"")
小开 
sent = "".join(e for e in sent if e.isalpha())
小开

使用ASCII可打印文件的随机字符串计时:

from inspect import getsource
from random import sample
import re
from string import printable
from timeit import timeit


pattern_single = re.compile(r'[\W]')
pattern_repeat = re.compile(r'[\W]+')
translation_tb = str.maketrans('', '', ''.join(c for c in map(chr, range(256)) if not c.isalnum()))




def generate_test_string(length):
return ''.join(sample(printable, length))




def main():
for i in range(0, 60, 10):
for test in [
lambda: ''.join(c for c in generate_test_string(i) if c.isalnum()),
lambda: ''.join(filter(str.isalnum, generate_test_string(i))),
lambda: re.sub(r'[\W]', '', generate_test_string(i)),
lambda: re.sub(r'[\W]+', '', generate_test_string(i)),
lambda: pattern_single.sub('', generate_test_string(i)),
lambda: pattern_repeat.sub('', generate_test_string(i)),
lambda: generate_test_string(i).translate(translation_tb),


]:
print(timeit(test), i, getsource(test).lstrip('            lambda: ').rstrip(',\n'), sep='\t')




if __name__ == '__main__':
main()

结果(Python 3.7):

       Time       Length                           Code
6.3716264850008880  00  ''.join(c for c in generate_test_string(i) if c.isalnum())
5.7285426190064750  00  ''.join(filter(str.isalnum, generate_test_string(i)))
8.1875841680011940  00  re.sub(r'[\W]', '', generate_test_string(i))
8.0002205439959650  00  re.sub(r'[\W]+', '', generate_test_string(i))
5.5290945199958510  00  pattern_single.sub('', generate_test_string(i))
5.4417179649972240  00  pattern_repeat.sub('', generate_test_string(i))
4.6772285089973590  00  generate_test_string(i).translate(translation_tb)
23.574712151996210  10  ''.join(c for c in generate_test_string(i) if c.isalnum())
22.829975890002970  10  ''.join(filter(str.isalnum, generate_test_string(i)))
27.210196289997840  10  re.sub(r'[\W]', '', generate_test_string(i))
27.203713296003116  10  re.sub(r'[\W]+', '', generate_test_string(i))
24.008979928999906  10  pattern_single.sub('', generate_test_string(i))
23.945240008994006  10  pattern_repeat.sub('', generate_test_string(i))
21.830899796994345  10  generate_test_string(i).translate(translation_tb)
38.731336012999236  20  ''.join(c for c in generate_test_string(i) if c.isalnum())
37.942474347000825  20  ''.join(filter(str.isalnum, generate_test_string(i)))
42.169366310001350  20  re.sub(r'[\W]', '', generate_test_string(i))
41.933375883003464  20  re.sub(r'[\W]+', '', generate_test_string(i))
38.899814646996674  20  pattern_single.sub('', generate_test_string(i))
38.636144253003295  20  pattern_repeat.sub('', generate_test_string(i))
36.201238164998360  20  generate_test_string(i).translate(translation_tb)
49.377356811004574  30  ''.join(c for c in generate_test_string(i) if c.isalnum())
48.408927293996385  30  ''.join(filter(str.isalnum, generate_test_string(i)))
53.901889764994850  30  re.sub(r'[\W]', '', generate_test_string(i))
52.130339455994545  30  re.sub(r'[\W]+', '', generate_test_string(i))
50.061149017004940  30  pattern_single.sub('', generate_test_string(i))
49.366573111998150  30  pattern_repeat.sub('', generate_test_string(i))
46.649754120997386  30  generate_test_string(i).translate(translation_tb)
63.107938601999194  40  ''.join(c for c in generate_test_string(i) if c.isalnum())
65.116287978999030  40  ''.join(filter(str.isalnum, generate_test_string(i)))
71.477421126997800  40  re.sub(r'[\W]', '', generate_test_string(i))
66.027950693998720  40  re.sub(r'[\W]+', '', generate_test_string(i))
63.315361931003280  40  pattern_single.sub('', generate_test_string(i))
62.342320287003530  40  pattern_repeat.sub('', generate_test_string(i))
58.249303059004890  40  generate_test_string(i).translate(translation_tb)
73.810345625002810  50  ''.join(c for c in generate_test_string(i) if c.isalnum())
72.593953348005020  50  ''.join(filter(str.isalnum, generate_test_string(i)))
76.048324580995540  50  re.sub(r'[\W]', '', generate_test_string(i))
75.106637657001560  50  re.sub(r'[\W]+', '', generate_test_string(i))
74.681338128997600  50  pattern_single.sub('', generate_test_string(i))
72.430461594005460  50  pattern_repeat.sub('', generate_test_string(i))
69.394243567003290  50  generate_test_string(i).translate(translation_tb)
< p > str.maketrans,str.translate是最快的，但包括所有非ascii字符。 re.compile,pattern.sub较慢，但以某种方式比''.join快。filter . < / p >
小开

如果我理解正确，最简单的方法是使用正则表达式，因为它为您提供了很大的灵活性，但另一个简单的方法是使用循环以下是示例代码，我还计算了单词的出现并存储在字典中。

s = """An... essay is, generally, a piece of writing that gives the author's own
argument — but the definition is vague,
overlapping with those of a paper, an article, a pamphlet, and a short story. Essays
have traditionally been
sub-classified as formal and informal. Formal essays are characterized by "serious
purpose, dignity, logical
organization, length," whereas the informal essay is characterized by "the personal
element (self-revelation,
individual tastes and experiences, confidential manner), humor, graceful style,
rambling structure, unconventionality
or novelty of theme," etc.[1]"""


d = {}      # creating empty dic
words = s.split() # spliting string and stroing in list
for word in words:
new_word = ''
for c in word:
if c.isalnum(): # checking if indiviual chr is alphanumeric or not
new_word = new_word + c
print(new_word, end=' ')
# if new_word not in d:
#     d[new_word] = 1
# else:
#     d[new_word] = d[new_word] +1
print(d)

如果这个答案是有用的，请评价这个!

小开

对于简单的一行代码(Python 3.0):

''.join(filter( lambda x: x in '0123456789abcdefghijklmnopqrstuvwxyz', the_string_you_want_stripped ))

Python <3.0:

filter( lambda x: x in '0123456789abcdefghijklmnopqrstuvwxyz', the_string_you_want_stripped )

注意:如果需要，您可以将其他字符添加到允许字符列表中(例如:“0123456789 abcdefghijklmnopqrstuvwxyz。_”)。

小开

Python 3

使用与@John Machin的回答相同的方法，但为Python 3更新:

  • 更大的字符集
  • 轻微改变translate的工作方式。

Python代码现在被假定为UTF-8编码
(来源:PEP 3120) < / p >

这意味着包含你想要删除的所有字符的字符串会变得更大:


del_chars = ''.join(c for c in map(chr, range(1114111)) if not c.isalnum())

translate方法现在需要使用一个可以用maketrans()创建的转换表:


del_map = str.maketrans('', '', del_chars)

现在，像以前一样，任何字符串s你想要&;卷曲&;


scrunched = s.translate(del_map)

使用@Joe Machin的最后一个计时示例，我们可以看到它仍然比re强一个数量级:


> python -mtimeit -s"d=''.join(c for c in map(chr,range(1114111)) if not c.isalnum());m=str.maketrans('','',d);s='foo-'*25" "s.translate(m)"
    

1000000 loops, best of 5: 255 nsec per loop
    

> python -mtimeit -s"import re;s='foo-'*25;r=re.compile(r'[\W_]+')" "r.sub('',s)"
    

50000 loops, best of 5: 4.8 usec per loop
小开

这是一个简单的解决方案，因为这里所有的答案都很复杂

filtered = ''
for c in unfiltered:
if str.isalnum(c):
filtered += c
    

print(filtered)
小开

我用perfplot(我的一个项目)检查了结果，发现对于短字符串，

"".join(filter(str.isalnum, s))

是最快的。对于长字符串(200+字符)

re.sub("[\W_]", "", s)

是最快的。

enter image description here

代码重现情节:

import perfplot
import random
import re
import string


pattern = re.compile("[\W_]+")




def setup(n):
return "".join(random.choices(string.ascii_letters + string.digits, k=n))




def string_alphanum(s):
return "".join(ch for ch in s if ch.isalnum())




def filter_str(s):
return "".join(filter(str.isalnum, s))




def re_sub1(s):
return re.sub("[\W_]", "", s)




def re_sub2(s):
return re.sub("[\W_]+", "", s)




def re_sub3(s):
return pattern.sub("", s)




b = perfplot.bench(
setup=setup,
kernels=[string_alphanum, filter_str, re_sub1, re_sub2, re_sub3],
n_range=[2**k for k in range(10)],
)
b.save("out.png")
b.show()
小开

如果你想保留像áéíóúãẽĩõũ这样的字符，使用这个:

import re
re.sub('[\W\d_]+', '', your_string)

提交答案